Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................X...............X........................] duration:
5.573977928s
FAIL (2025-10-19 02:02:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.574014137s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....................X...............X.......................] duration:
5.737126047s
FAIL (2025-10-19 02:01:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.737158549s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................X...............X......................] duration:
5.612452581s
FAIL (2025-10-19 02:00:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.612489072s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................X...............X.....................] duration:
5.03613143s
FAIL (2025-10-19 01:59:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.036176094s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................X...............X....................] duration:
5.071949765s
FAIL (2025-10-19 01:58:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.07197312s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................X...............X...................] duration:
5.025209828s
FAIL (2025-10-19 01:57:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.025243162s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................X...............X..................] duration:
5.058948958s
FAIL (2025-10-19 01:57:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.058979544s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................X...............X.................] duration:
5.079627234s
FAIL (2025-10-19 01:56:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.079660919s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................X...............X................] duration:
5.15469531s
FAIL (2025-10-19 01:55:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.15473198s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............................X...............X...............] duration:
5.12415026s
FAIL (2025-10-19 01:54:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.124188252s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............................X...............X..............] duration:
5.000735339s
FAIL (2025-10-19 01:53:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.000766407s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..............................X...............X.............] duration:
5.069381457s
FAIL (2025-10-19 01:52:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.06940914s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...............................X...............X............] duration:
5.049347973s
FAIL (2025-10-19 01:52:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.049382268s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [................................X...............X...........] duration:
5.031557518s
FAIL (2025-10-19 01:51:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.031587205s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.................................X...............X..........] duration:
5.085135619s
FAIL (2025-10-19 01:51:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.085215551s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..................................X...............X.........] duration:
5.055877259s
FAIL (2025-10-19 01:49:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.055911283s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................................X...............X........] duration:
5.076749352s
FAIL (2025-10-19 01:49:09 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.076785941s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....................................X...............X.......] duration:
5.069185146s
FAIL (2025-10-19 01:48:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.069215422s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X...............X......] duration:
5.050283626s
FAIL (2025-10-19 01:47:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.050318592s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................................X...............X.....] duration:
5.056626422s
FAIL (2025-10-19 01:46:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.056660137s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X...............X....] duration:
5.049177286s
FAIL (2025-10-19 01:45:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.049210249s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................................X...............X...] duration:
5.074735427s
FAIL (2025-10-19 01:45:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.074768289s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X...............X..] duration:
5.14341413s
FAIL (2025-10-19 01:45:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 5.143451541s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................................X...............X.] duration:
4.939986794s
FAIL (2025-10-19 01:44:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 4.940012984s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................................X...............X] duration:
1m0.195874795s
FAIL (2025-10-19 01:44:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 21)
======LOGS======
6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.901877637 (612h11m47.363970141s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.902841002 (612h11m47.364933485s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.903737900 (612h11m47.365830383s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.904479121 (612h11m47.366571605s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:01.905250921 (612h11m47.367343404s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Uploading piece...
Oct 19 01:43:02.738978060 (612h11m48.201070553s) [upload=538603]: Next returned nil segment
Oct 19 01:43:02.739007296 (612h11m48.201099779s) [upload=538603]: Waiting for error group managing segments...
Oct 19 01:43:03.109585662 (612h11m48.571678135s) [noise=true , address=81.171.14.70:4029 , piece_id=J3XQYJYQKFM6GWRQSABDIEZWTQQ7KVT26TXQ7JFAFCXB3MRHT3KQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.118574738 (612h11m48.580667221s) [noise=true , address=81.171.14.70:4036 , piece_id=2UVLZFIP3LMYIZN6ECJV4YMSIFS2DI7UWETGBZMKQUTZK566QWUA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.170576547 (612h11m48.63266903s) [noise=true , address=89.149.201.147:4020 , piece_id=EPRH3OQA622QKSF2QIIBGWVOXLIOSUKCOL5COKPXTEOZQO35PRHQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.171732098 (612h11m48.633824591s) [noise=true , address=81.171.14.70:4044 , piece_id=IEKVUIMT3J4OHCGS76B567G2SGLIWEUECMPS6SQG65GGVOFSGZXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.173759678 (612h11m48.635852162s) [noise=true , address=81.171.14.70:4045 , piece_id=7BA2WX2RSN7MTYPQ7JVNO64YQJ73OSZPJTBYFBXQZW3CYCCSI32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.182275905 (612h11m48.644368388s) [noise=true , address=89.149.201.147:4003 , piece_id=HZCXTDRF3XSKOO553YVKCJCCYQZUAAQVB4HDS4B67IWRNYU5TQQA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.198367886 (612h11m48.660460369s) [noise=true , address=89.149.201.147:4005 , piece_id=IEJNUW7DOEEL2GMK4L4FNZBWF4XISJU6HSRQB6AI3O27UFBCP32Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.206915922 (612h11m48.669008405s) [noise=true , address=81.171.14.70:4039 , piece_id=2OE3OT5LGJIAPLSWMKLB7GWFKWJ4PJLDBXDKQXAEHTHCMQ34ZJPQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.211060986 (612h11m48.673153469s) [noise=true , address=89.149.201.147:4017 , piece_id=LUGEBDJMR6LJWT53Y2GRZMXPBEJIMX4DIWH6MMP5WEKEAWMDPUOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.221278421 (612h11m48.683370904s) [noise=true , address=89.149.201.147:4009 , piece_id=3AUY3MV23ZVXH2GJ3KQJZQ3SKH2LLDYV6XBBBZ3F3CJKBHQF276A, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.235331901 (612h11m48.697424394s) [noise=true , address=89.149.201.147:4019 , piece_id=TXFSVW37GMQQHIF47I65LN3POR5MIMZN774KFVQQU5L6GQD3NKIA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.250710585 (612h11m48.712803068s) [noise=true , address=89.149.201.147:4018 , piece_id=T5Q2ABQRBPX5FOGU77ZVUDMJHKP3XMCCTECNPCKZ4NCONIAOZW4Q, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.664781326 (612h11m49.126873829s) [noise=true , address=81.171.14.70:4032 , piece_id=H5QSA6PIFHOQFWHTGXSJA6SG54FB4LTEPKOQFEPOZ7AQKE4TTQXQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:03.697750314 (612h11m49.159842797s) [noise=true , address=89.149.201.147:4022 , piece_id=XGOQJZG5V4XQY77QNQXNXMZ7ZVKJWFCZDVPW5UP7FG7GITIVQNNQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:53.943361631 (612h12m39.405454104s) [noise=true , address=185.167.204.206:4057 , piece_id=HFMODSHOBOK6VLVEISZ4OPKZ7OU3ZKYHY7ARXMNMYTMGLRIXGIMQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:54.493680810 (612h12m39.955773293s) [noise=true , address=185.167.204.206:4062 , piece_id=6S34CYPKHQDKNK52RSE7ZHOCFB5NOUA3WOQUPITMV7BTYWVCV5QQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.128831680 (612h12m40.590924153s) [noise=true , address=185.167.204.206:4071 , piece_id=GLIT2XIFWARZKNVLUVLZVW7QQH6MVVWNMDJI6JIGAFUO5JRGCPWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:55.183040983 (612h12m40.645133466s) [noise=true , address=185.167.204.205:4010 , piece_id=OWDTDV3WADVG4RUT2PLCASAZQ3BEQX3PAWID6M74CMZ4I5TRXHAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:56.953384888 (612h12m42.415477371s) [noise=true , address=185.167.204.206:4064 , piece_id=QHWYEJD5GIRCWLYMEXUWVFAHZJIE2Y36SJ7UN25EOLJJOHDTIXAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:43:58.385933992 (612h12m43.848026465s) [noise=true , address=185.167.204.205:4022 , piece_id=NHHX44F7ZKZ6I5H7PAWIPK4Z2QDPUHOCPDQYR74WWLMOROT2DESA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:00.210575548 (612h12m45.672668031s) [noise=true , address=185.167.204.205:4006 , piece_id=4V4EAGG6WY26OTSKAKDPAR5DHMSUOMDEJ6OVHV4IH24PBIL5CCWQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: <nil>
Oct 19 01:44:01.154637923 (612h12m46.616730406s) [noise=true , address=185.167.204.206:4041 , piece_id=WMEYGLWNQPJ72ALTEH3SENPTBT7KU4WDGGKA7RYZ34KPQMLW4DAQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, address:185.167.204.206:4041): protocol: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33322->185.167.204.206:4041: use of closed network connection
Oct 19 01:44:01.154684291 (612h12m46.616776764s) [noise=true , address=185.167.204.205:4034 , piece_id=OXBG6B5EJG2GXHG7YV2OXCXTR2DWIL3EMKYSTRRSJDPVXPOS3YNA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1pRmbsqHeJVQz9zeLEtzcfb7gYRNHFDXHRmqNeApdPkcNX7Wur, address:185.167.204.205:4034): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154807596 (612h12m46.616900079s) [noise=true , address=185.167.204.205:4017 , piece_id=KBVJPHJDL67UF2FA4VDEOY7AQGTJLJ46F7FBKCNVRUD3EUSXT4CQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12em75cRFPaX8jTLrVV4sbuL4AgzzHopjFfiJ6MsvvYKKGNFe8h, address:185.167.204.205:4017): protocol: expected piece hash; context deadline exceeded; EOF
Oct 19 01:44:01.154819779 (612h12m46.616912262s) [noise=true , address=185.167.204.206:4038 , piece_id=MDA4GRHS5WOIAF7C5TFVI2CNQXSMPILZMNQHLTLPXZCMTAUE6IXA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:1jtjiXHQRFhyw6vb8A1RDPfe4j2283znmAZiZCBmrvFpQrqJiT, address:185.167.204.206:4038): protocol: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41700->185.167.204.206:4038: use of closed network connection
Oct 19 01:44:01.154891064 (612h12m46.616983548s) [noise=true , address=185.167.204.205:4029 , piece_id=NH2I5G7U32ACNB7OYSJ7BBELS7GVSM3JQWTGBZMPH5CWSMIXBZOA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12DyF12w2JXdkGLfLx9VE1E5f28VnMg7M2b3osfsDtGX9o8Tvsn, address:185.167.204.205:4029): protocol: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55414->185.167.204.205:4029: use of closed network connection
Oct 19 01:44:01.154953343 (612h12m46.617045816s) [noise=true , address=185.167.204.205:4001 , piece_id=IXNCO2UNTHJ4P7YYMBQXQ5B3RJBHNWZRR7WKGO5KDTUARSASVNDQ, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12gZXkE3Cnmu2oyKMmUfaADZ4LDPQahTcE9sUwW173on719Bb91, address:185.167.204.205:4001): protocol: read tcp 188.166.225.170:57742->185.167.204.205:4001: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155063803 (612h12m46.617156277s) [noise=true , address=185.167.204.205:4018 , piece_id=ERVNBV6WQ6QUC2BBT3SOKQUOWWUU6PF6WIY5CVC6VYMWTRLX4KSA, seg_pos={0 0}, upload=538603]: Done uploading piece. err: ecclient: upload failed (node:12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, address:185.167.204.205:4018): protocol: read tcp 188.166.225.170:45168->185.167.204.205:4018: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 19 01:44:01.155093450 (612h12m46.617185933s) [seg_pos={0 0}, upload=538603]: Done waiting for segment. successful: 21 optimal: 26 errs: <nil>
Oct 19 01:44:01.347539156 (612h12m46.809631649s) [upload=538603]: Done uploading segments...
Oct 19 01:44:01.347573822 (612h12m46.809666305s) [upload=538603]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 21)
Duration: 1m0.195906945s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.674408872s
FAIL (2025-10-19 01:43:10 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.67446002s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
8.517443438s
FAIL (2025-10-19 01:42:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 8.51747638s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
5.770516334s
FAIL (2025-10-19 01:42:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.771101769s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.131671421s
FAIL (2025-10-19 01:42:08 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.13171354s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.060227678s
FAIL (2025-10-19 01:41:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.060257675s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.64326422s
FAIL (2025-10-19 01:41:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.643291712s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.031691189s
FAIL (2025-10-19 01:41:14 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.031723321s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
5.717388665s
FAIL (2025-10-19 01:40:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.717547648s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.072095279s
FAIL (2025-10-19 01:40:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.072129104s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.885674994s
FAIL (2025-10-19 01:40:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.885699429s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.077339694s
FAIL (2025-10-19 01:39:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.077360574s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.628656754s
FAIL (2025-10-19 01:39:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.628678745s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.042105199s
FAIL (2025-10-19 01:38:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.042134917s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.785219243s
FAIL (2025-10-19 01:38:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.78524917s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.020778666s
FAIL (2025-10-19 01:38:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.020810847s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.853903101s
FAIL (2025-10-19 01:37:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.853976209s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.005910224s
FAIL (2025-10-19 01:37:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.005940601s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.075871804s
FAIL (2025-10-19 01:36:58 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.075906339s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.00705185s
FAIL (2025-10-19 01:36:38 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.00708392s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
5.707234935s
FAIL (2025-10-19 01:36:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.70725364s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
4.997260205s
FAIL (2025-10-19 01:35:55 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 4.99731015s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
5.768452918s
FAIL (2025-10-19 01:35:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.768483585s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.165151415s
FAIL (2025-10-19 01:35:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 5.165188486s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
4.979503677s
FAIL (2025-10-19 01:34:27 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 12FDxcFVTdQeudwXi3bRkqnBSQ7NhjChgqkCZLmEC9T4bxdGA4w, Piece ID: N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA) context deadline exceeded; piecestore: (Node ID: 12B18LegNdvxjRqBgwnFB4Aig1nt5URm9Au7bWXsfbBPyCnterM, Piece ID: 6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ) context deadline exceeded; piecestore: (Node ID: 1uSTuNA28oH2s5Rkv2jaKoC2nZWSgpMJXyxwFTCKbr44EvEXrC, Piece ID: VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A) context deadline exceeded; piecestore: (Node ID: 12ZHTTaGQUTftxEULFqWbH6w1HnWA7PavHrXnvMCD947FxtmsrS, Piece ID: UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ) context deadline exceeded; piecestore: (Node ID: 12khRPUvAHZgwi8rajx8TYpzujWybn3tcwZeG81R7JzHFRGL8PS, Piece ID: SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ) context deadline exceeded; piecestore: (Node ID: 12hK2MSVcFNaCVp7k4krUva7jXEJYhjnHPxPrMgq87jcDXJPkv2, Piece ID: UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A) context deadline exceeded; piecestore: (Node ID: 1m3rag6G75ZAJkXs6tMaHceq1e2mxp2LrYW8zct4p1Ra3pNhaC, Piece ID: C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA) context deadline exceeded; piecestore: (Node ID: 12w2LyfAtY3ihjJgkBtord4fMMfbEvz73nExjTNZhLQXurdA3iS, Piece ID: 7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ) context deadline exceeded; piecestore: (Node ID: 12mQUrzCLU4imW5BNBpkSWnRtUftKav85Xi2RsEBacP31AXJ7R8, Piece ID: IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ) context deadline exceeded; piecestore: (Node ID: 1216DSoBdbhQNEkdg1TtfmYRJrUjLxCHKvdkhB1axUiaDNXGrrj, Piece ID: MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q) context deadline exceeded
======LOGS======
VLMQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.937137175 (612h0m11.399229658s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942046182 (612h0m11.404138655s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.942418159 (612h0m11.404510642s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.944875037 (612h0m11.40696754s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.947909184 (612h0m11.410001667s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.949316424 (612h0m11.411408907s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.951308036 (612h0m11.413400519s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.955818282 (612h0m11.417910755s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:25.957565077 (612h0m11.419657551s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Uploading piece...
Oct 19 01:31:26.788962703 (612h0m12.251055176s) [upload=538431]: Next returned nil segment
Oct 19 01:31:26.788984594 (612h0m12.251077067s) [upload=538431]: Waiting for error group managing segments...
Oct 19 01:31:27.206621086 (612h0m12.668713559s) [noise=true , address=81.171.14.70:4026 , piece_id=3L6ZFCW6B63IXALVFY3ZXVYLNRGTPNDE3IA6A65ZTR4QSSWOXUQA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.215893780 (612h0m12.677986264s) [noise=true , address=81.171.14.70:4031 , piece_id=UENTSWVLOKQTKFTRZYK4YGEDINUF4THWFGOKFOAEKGL3XKWY5XUQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.218043203 (612h0m12.680135686s) [noise=true , address=89.149.201.147:4003 , piece_id=BDRQCX6DZMKS3LHOVPOCEZTLEQBQPVWQBR64LQVP34E7Q2HT55EA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.221631676 (612h0m12.683724159s) [noise=true , address=81.171.14.70:4025 , piece_id=VDGTACTRVWHIH5HY4LZ2JR7IMQF3EGLHTUKPWLG2ABOVMJ5ENMJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.228228334 (612h0m12.690320817s) [noise=true , address=89.149.201.147:4005 , piece_id=C2HBG4HF6P22DKO32F4WV2ZWGDOZ2NRO2KHMBR2LX52IUAS7VLMQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.229336304 (612h0m12.691428777s) [noise=true , address=81.171.14.70:4046 , piece_id=KMZ34CV4DZHYCJP767J7QMDE3S3B3LYFQF2LSOJMVUEQUVFKVQRQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.237726719 (612h0m12.699819202s) [noise=true , address=89.149.201.147:4007 , piece_id=BTL7WMQYSO2GQIGCIT2BAUATSIAQMRSZKW6YRIPDXW5K7P6TNJ5Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239706107 (612h0m12.70179859s) [noise=true , address=81.171.14.70:4039 , piece_id=IFHIQUY6IY2JDMWVCIW3IBTRLEXLRY7HNZSMF7HJRYFVULJQZPJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.239756322 (612h0m12.701848815s) [noise=true , address=81.171.14.70:4047 , piece_id=SJHXW2SJF4QIKNBX3X3VIPG2NS6NZFCBBBIKYQUGR46LSXDFABEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.260059217 (612h0m12.72215169s) [noise=true , address=89.149.201.147:4017 , piece_id=AMY5AKLGQ5A5TXHKK7ION3U7LOIUO3JLN36ZV6TWIUHT7VEGUVJA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.323909964 (612h0m12.786002447s) [noise=true , address=89.149.201.147:4024 , piece_id=WNKTUGVQVKTAHLZBXQ7T5GW3UJBFDUIHHBVVSE3LQF4NE5SGSC2A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.675889661 (612h0m13.137982144s) [noise=true , address=89.149.201.147:4016 , piece_id=QLBPR7PI5PK5YYQ6CLOQ5KY4NQC65NOOUKDE4N5VUHDIT4TBRL3A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.693461496 (612h0m13.155553979s) [noise=true , address=89.149.201.147:4011 , piece_id=ZK4L3O5IOJF7AHGIRXWF7A5FTFN3HVJQLIAKMMX63EKVSJONRFSA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.713290149 (612h0m13.175382622s) [noise=true , address=81.171.14.70:4040 , piece_id=SW7EOQP6CSZU5K755PHB3QKW2MCEG2EHMUF7EYWC3JFTPA2P6M6A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:31:27.723999157 (612h0m13.18609163s) [noise=true , address=89.149.201.147:4015 , piece_id=B4KFHM62WWC4TQGSJ72QXBQSABZESTNOONAORXQUCAS5B625AXDA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:11.170266279 (612h0m56.632358752s) [noise=true , address=185.167.204.205:4021 , piece_id=N3ZZTMSA72OLYHEYOFZCXWSLBM6SQRSJK4DUJ3WQ6T2EUOIOH7JA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:13.938416283 (612h0m59.400508767s) [noise=true , address=185.167.204.205:4005 , piece_id=6DFRCB33NJHTGHGPZDHEE5EQEXTP2P5VIROJA757OSBNHQVAV6GQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:15.180763204 (612h1m0.642855688s) [noise=true , address=185.167.204.206:4056 , piece_id=VYEGOWQI5ZOVIAIRSQJCKWU2SSAXDH7INVMYNYBGSYG3STM3666A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.108438663 (612h1m1.570531146s) [noise=true , address=185.167.204.205:4018 , piece_id=7ALB2Q6ZQLWEPIAPEPURDWUZRSDYES57JVULZIEUPWTMKGB6IIFQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.158571772 (612h1m1.620664255s) [noise=true , address=185.167.204.206:4041 , piece_id=C65LSECERH4QSZ42XM5QN32HRLRCIDLR3ZHAV35O3HJJFOM4JHEA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.492153591 (612h1m1.954246064s) [noise=true , address=185.167.204.206:4051 , piece_id=UW5UFGNV6DLGXJ2GSGIUSHG6RFOQZD3MX74GSPSBZBKMW6HCWA7A, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:16.659732987 (612h1m2.12182547s) [noise=true , address=185.167.204.205:4006 , piece_id=IM23EOM2EDZW24LGJDRAMXVRLUTGNX6SZZZGUCYRQNHDGUY3TBOQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:17.685252631 (612h1m3.147345114s) [noise=true , address=185.167.204.205:4008 , piece_id=UKF7VBVWLGU5WE76O67Y5W3Y5ATUXGVUKQ66WDF3WS3MR4R744CQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.060327770 (612h1m3.522420243s) [noise=true , address=185.167.204.206:4067 , piece_id=SFATJEPASZR5C3OTJ6KVJ6OQ65OE3KQ537BJF62A2464PBXHO5HQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:18.149921018 (612h1m3.612013501s) [noise=true , address=185.167.204.205:4001 , piece_id=DAX6HSS3SLHSW42HXWUIJL2MS7YGFFPWDJWWOERWFTRFSHVTNAGA, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597750295 (612h1m9.059842778s) [noise=true , address=185.167.204.206:4044 , piece_id=MAKWTGL6TPLFZY3V5B2D5MOFUGGES3E7OIAJUBNUFN3MSJ2ABT4Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: <nil>
Oct 19 01:32:23.597786384 (612h1m9.059878867s) [seg_pos={0 0}, upload=538431]: Segment reached optimal threshold of 26 pieces.
Oct 19 01:32:23.597948733 (612h1m9.060041227s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xy38NywWy5CSJ7CfupY78vbkHg6cQzgNhM2Z1xnpvw9dwvMoT): protocol: expected piece hash; context canceled; EOF
Oct 19 01:32:23.597957320 (612h1m9.060049793s) [noise=true , address=185.167.204.206:4057 , piece_id=CISG2VEIEZUO53HIXAJ4WGI23SBZ4GG4DJ4S4MPDEA4XKZEA533Q, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598082609 (612h1m9.060175082s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZQ9gXppcqJGQ66ZAu1rm7QHdkJHaLcE3dXtBFiHC68Kffm8Hm): protocol: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60652->185.167.204.205:4004: use of closed network connection
Oct 19 01:32:23.598100582 (612h1m9.060193065s) [noise=true , address=185.167.204.205:4004 , piece_id=26OPDPNNC2T5ZTLZYRDJLOE5DNRHAU46HZXM3FVVDAJFKMMD5FZQ, seg_pos={0 0}, upload=538431]: Long tail cancellation due to optimal threshold
Oct 19 01:32:23.598118116 (612h1m9.060210599s) [seg_pos={0 0}, upload=538431]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Oct 19 01:32:23.598210471 (612h1m9.060302954s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 19 01:32:23.598215360 (612h1m9.060307844s) [upload=538431]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 19 01:32:23.877058122 (612h1m9.339150605s) [upload=538431]: Done uploading segments...
Oct 19 01:32:23.877079273 (612h1m9.339171746s) [upload=538431]: Upload finished. err: <nil>
Duration: 4.979535417s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
5.768195895s
FAIL (2025-10-19 01:34:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.768220723s
core
uplink
us-select-1
us1-placement-20
|