Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.786001339s
FAIL (2025-10-08 21:05:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.786040163s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
4.958543624s
FAIL (2025-10-08 21:04:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.95857771s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
6.773842759s
FAIL (2025-10-08 21:04:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.773920418s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
6.564535174s
FAIL (2025-10-08 21:04:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.564571553s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
4.825885397s
FAIL (2025-10-08 21:03:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.825928328s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.372927753s
FAIL (2025-10-08 21:03:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.372964263s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.080660333s
FAIL (2025-10-08 21:03:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.08071094s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
4.793810347s
FAIL (2025-10-08 21:02:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.793844262s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
6.229785008s
FAIL (2025-10-08 21:01:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.229808553s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.856038394s
FAIL (2025-10-08 21:01:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.856067731s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.172227691s
FAIL (2025-10-08 21:01:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.17225386s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
5.681668436s
FAIL (2025-10-08 21:00:53 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.681704725s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
6.215877602s
FAIL (2025-10-08 21:00:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.2159127s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.870715682s
FAIL (2025-10-08 21:00:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.870828497s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.094680274s
FAIL (2025-10-08 21:00:10 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.094712886s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.044427753s
FAIL (2025-10-08 21:00:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.044459793s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.176382202s
FAIL (2025-10-08 20:59:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.176416468s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
4.834158829s
FAIL (2025-10-08 20:59:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.834195279s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.033041401s
FAIL (2025-10-08 20:59:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 5.033074213s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
6.343133599s
FAIL (2025-10-08 20:58:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.343166031s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
4.862935795s
FAIL (2025-10-08 20:58:40 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 4.862959641s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.171175174s
FAIL (2025-10-08 20:58:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 24)
======LOGS======
2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571113087 (367h26m21.033205561s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571581779 (367h26m21.033674263s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571664478 (367h26m21.033756961s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.571950522 (367h26m21.034042995s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572391872 (367h26m21.034484375s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.572804458 (367h26m21.034896931s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573198298 (367h26m21.035290781s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.573694723 (367h26m21.035787196s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:35.574277153 (367h26m21.036369636s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Uploading piece...
Oct 8 20:57:36.415571773 (367h26m21.877664246s) [upload=323246]: Next returned nil segment
Oct 8 20:57:36.415592041 (367h26m21.877684514s) [upload=323246]: Waiting for error group managing segments...
Oct 8 20:57:36.837364258 (367h26m22.299456741s) [noise=true , address=89.149.201.147:4024 , piece_id=TODEAZ4NVIK4MW36DRSQRLGJEMZMJ5R7BO5KUUQEIXMRFCLG7QYA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.855429558 (367h26m22.317522042s) [noise=true , address=89.149.201.147:4011 , piece_id=MZM6VXS7HSQYR6UK6FCZKUBS2BDWEO4FLZJWRK344PKWRIY6XWBA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.879810423 (367h26m22.341902896s) [noise=true , address=89.149.201.147:4017 , piece_id=CR5MOJK4DFWUOPE6ESIQKZ47WE2JZXESENQJOMDTXM55KHJOOELQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.883723565 (367h26m22.345816048s) [noise=true , address=89.149.201.147:4006 , piece_id=3YMNW3HN56X3TI5SRWJYWV3URJQMPLO3IWSG6PCJBIM3RKE2T6GQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.901117657 (367h26m22.36321014s) [noise=true , address=89.149.201.147:4004 , piece_id=RMNY3M6SFODMCHWBGVMM3IDAZZG656IEN7A66U3TIZRC73MYZ4RA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:36.932843199 (367h26m22.394935682s) [noise=true , address=89.149.201.147:4022 , piece_id=WAT74AALSY6EGQ7MJWBHOBFRQD5446XPJUZLRWWF7X63KUXILQ7Q, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.360065203 (367h26m22.822157686s) [noise=true , address=185.167.204.206:4071 , piece_id=FM4ALMFGB5L3Y2L4OMXVNPDFLUIRZS2OW5MKVHKHDQQRVSFOMF6A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.384563773 (367h26m22.846656246s) [noise=true , address=185.167.204.205:4010 , piece_id=ZJM6KDQIDIDITXMQSPQJESRYW44CM3SUEOSFMAUPBQQIN47IGAEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.391690524 (367h26m22.853783007s) [noise=true , address=185.167.204.205:4002 , piece_id=L4IMLN45IXL2RAEOLBPXTHLCNDH7MP73BKJ57YD6N6W5CBNTK65A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.400827563 (367h26m22.862920046s) [noise=true , address=185.167.204.206:4059 , piece_id=42T7XTJREXHLTSNMT23GEQKRUZCDDI4LBIKP5PRRCT7FHUBYK6XQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.408014649 (367h26m22.870107122s) [noise=true , address=185.167.204.206:4038 , piece_id=KNNAH4Q4VPMOXSX5UGZDYAOCH6EZRQXCXOAYS345VV4JXEICKUJA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.415024657 (367h26m22.87711714s) [noise=true , address=185.167.204.205:4026 , piece_id=5JLBBZ3YQ6YBROI53NHYRTSTZ7DYZE4X7LWXPPRPWHFIK3FWRXCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.417851380 (367h26m22.879943873s) [noise=true , address=185.167.204.205:4030 , piece_id=TOXOLL2MGUMAL4UNDJMP3H6KAARLNYG6AF5SPYHRLDWGGE5S5MUA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.458107545 (367h26m22.920200018s) [noise=true , address=185.167.204.205:4015 , piece_id=WSR73QWVBGGD2U2RGLQNQFEJOO4YEQXVACGHPNSBG7HHICP73QBQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.466480670 (367h26m22.928573153s) [noise=true , address=185.167.204.206:4040 , piece_id=WJEF7QN5LOBDQUA5HIWCR6D3C5E6WCPYDC2SSZM4P2AR5QRDC3QA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.483345484 (367h26m22.945437967s) [noise=true , address=185.167.204.205:4022 , piece_id=BYMGJHVX7BTCXSGYGK7F7UVCAV4GWVTGSIPXYW2CDMD3ECROYKQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.508162991 (367h26m22.970255474s) [noise=true , address=185.167.204.206:4072 , piece_id=75XRMOTKRZURGO4P2ADZFJT5NPFX2MVRWV6DNVZUQUJY4PZQNJGQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.509282713 (367h26m22.971375196s) [noise=true , address=185.167.204.206:4064 , piece_id=REMLCRNQQIIBCG3YHFWT26754PZK5FUYSX3EWX2Y3T6SHEYRFHVA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.512045113 (367h26m22.974137586s) [noise=true , address=185.167.204.205:4020 , piece_id=JKDHELYFFIJ2FSDL3DKZ5TUK4NF6GGB3JUVHG3M4734IAGA6TKCQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:57:37.529867210 (367h26m22.991959683s) [noise=true , address=185.167.204.206:4046 , piece_id=DJXF6WOBDZEUMKSTLN7BBJN7NL3YPHUD7V5LQJLQHOAB4GAJQRPA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.366437718 (367h27m17.828530191s) [noise=true , address=81.171.14.70:4041 , piece_id=JMQOVVTAEDM6QCWMQHNKZLBIH4I23X35PTRRULBNQWIK2KDUHG7A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.457584630 (367h27m17.919677113s) [noise=true , address=81.171.14.70:4042 , piece_id=SDKNNU2CRWGSO2A5EIBJVYCGKZO2AFF7GZ2SONY6THUCWS6ZUXEQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:32.754826238 (367h27m18.216918721s) [noise=true , address=81.171.14.70:4043 , piece_id=AT6GA3PPHJZVMMRQB3R2ZROJT6VMM5RQ5EX525DOQNNK5LNNVL5A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:33.927248860 (367h27m19.389341343s) [noise=true , address=81.171.14.70:4045 , piece_id=JRA5QBLDWGRFRHZ3Y27ATPR2FLW6WN7CBWXGADM57JYRN227S4GA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: <nil>
Oct 8 20:58:34.888563372 (367h27m20.350655855s) [noise=true , address=81.171.14.70:4030 , piece_id=Y7XZNITOE2NP56F26PJEX2ZDGDD6MRY6RMHCSFESH2JGH3KZOAQQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45860->81.171.14.70:4030: use of closed network connection
Oct 8 20:58:34.888770096 (367h27m20.350862589s) [noise=true , address=81.171.14.70:4029 , piece_id=RQ4R3ETYF3AN2IS2XM6JIO624GE553SORPW7AQDLAA7T5EI4OYSQ, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:18yEfU2X4jSrom1JDtQ22cUEvomcg91ohvay2Nsr1Vr8juo9to, address:81.171.14.70:4029): protocol: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38712->81.171.14.70:4029: use of closed network connection
Oct 8 20:58:34.888886196 (367h27m20.350978669s) [noise=true , address=81.171.14.70:4038 , piece_id=WD6RZZJZDZ4TOUWNIWGIL7TCN3H2MZKYR3I35UOBHAC3YPXUWXWA, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:15im1ewAFZd3JR1JxHcRxcrSkGaM7VwporX63EoYSfPmmj5ZzB, address:81.171.14.70:4038): protocol: rpc: dial tcp 81.171.14.70:4038: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888930961 (367h27m20.351023444s) [noise=true , address=81.171.14.70:4033 , piece_id=FSNJ6F5PGR6YCKQPPQ5MUYYCDEWA2EDAEMYED35HQ25VLV7HM75A, seg_pos={0 0}, upload=323246]: Done uploading piece. err: ecclient: upload failed (node:1BnjusiXEfHYwTDL8soMkm8GYnhbkwy2oCAxVA7oiDyhFXozZ8, address:81.171.14.70:4033): protocol: read tcp 188.166.225.170:50980->81.171.14.70:4033: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:58:34.888961780 (367h27m20.351054263s) [seg_pos={0 0}, upload=323246]: Done waiting for segment. successful: 24 optimal: 26 errs: <nil>
Oct 8 20:58:35.056795929 (367h27m20.518888412s) [upload=323246]: Done uploading segments...
Oct 8 20:58:35.056828541 (367h27m20.518921024s) [upload=323246]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 24)
Duration: 1m0.171202797s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
6.232783975s
FAIL (2025-10-08 20:58:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.232806809s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
6.296483402s
FAIL (2025-10-08 20:58:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.296521534s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
6.101517579s
FAIL (2025-10-08 20:57:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.101552506s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
6.279100991s
FAIL (2025-10-08 20:56:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.279147169s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
6.396195915s
FAIL (2025-10-08 20:55:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.396228107s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
6.334464819s
FAIL (2025-10-08 20:54:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.334499064s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
6.343883293s
FAIL (2025-10-08 20:53:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.343991109s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.134798766s
FAIL (2025-10-08 20:52:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.134821468s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
6.212510242s
FAIL (2025-10-08 20:51:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.212546409s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
6.239156512s
FAIL (2025-10-08 20:50:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.239188814s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.168623603s
FAIL (2025-10-08 20:49:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.168658599s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
6.447678781s
FAIL (2025-10-08 20:49:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.447711613s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
6.327430755s
FAIL (2025-10-08 20:48:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.327465039s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
6.161899526s
FAIL (2025-10-08 20:47:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.161935966s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
6.147152319s
FAIL (2025-10-08 20:46:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.147188316s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
6.32820767s
FAIL (2025-10-08 20:45:37 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.328240873s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
6.206113104s
FAIL (2025-10-08 20:45:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.206147299s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
7.034580078s
FAIL (2025-10-08 20:44:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 7.034611928s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.247166369s
FAIL (2025-10-08 20:44:36 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.247196087s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.383762633s
FAIL (2025-10-08 20:43:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.38379245s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.995225053s
FAIL (2025-10-08 20:42:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.995307982s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.42678322s
FAIL (2025-10-08 20:41:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.426819872s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
15.037979225s
FAIL (2025-10-08 20:41:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 15.038000757s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
23.848587405s
FAIL (2025-10-08 20:40:49 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 23.848623714s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
7.557399907s
FAIL (2025-10-08 20:39:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 7.557432408s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
6.535578229s
FAIL (2025-10-08 20:39:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 6.535610811s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
7.071182618s
FAIL (2025-10-08 20:38:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 7.07121526s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.926355572s
FAIL (2025-10-08 20:37:46 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 11)
======LOGS======
8 20:20:43.719346347 (366h49m29.18143882s) [upload=322699]: Done starting upload
Oct 8 20:20:43.719535055 (366h49m29.181627549s) [upload=322699]: Uploading segments...
Oct 8 20:20:43.719698537 (366h49m29.18179104s) [upload=322699]: Got next segment. Inline: false
Oct 8 20:20:43.719762920 (366h49m29.181855413s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 8 20:20:43.719769973 (366h49m29.181862456s) [upload=322699]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 8 20:20:43.979669033 (366h49m29.441761516s) [seg_pos={0 0}, upload=322699]: Begin upload segment...
Oct 8 20:20:43.979752972 (366h49m29.441845445s) [seg_pos={0 0}, upload=322699]: Done begin upload segment.
Oct 8 20:20:43.979802948 (366h49m29.441895431s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979844647 (366h49m29.44193712s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.979936342 (366h49m29.442028825s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.980467563 (366h49m29.442560046s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981041295 (366h49m29.443133778s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981419726 (366h49m29.443512209s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.981788919 (366h49m29.443881402s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982218337 (366h49m29.44431082s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.982661821 (366h49m29.444754304s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.983128590 (366h49m29.445221063s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.985194805 (366h49m29.447287278s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.986668952 (366h49m29.448761435s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.988866175 (366h49m29.450958658s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.990750794 (366h49m29.452843277s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.992455961 (366h49m29.454548444s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:43.994289653 (366h49m29.456382136s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Uploading piece...
Oct 8 20:20:44.248655796 (366h49m29.710748289s) [noise=true , address=77.90.31.12:4592 , piece_id=J6ZKADCIXBJYGB5RWEE3QDMONRJDARRQ6B62WT22LZGL3LZBRVMQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3, address:77.90.31.12:4592): protocol: rpc: dial tcp 77.90.31.12:4592: connect: connection refused; conn closed: op; piecestore: piecestore close: conn closed: op
Oct 8 20:20:45.892543084 (366h49m31.354635567s) [upload=322699]: Next returned nil segment
Oct 8 20:20:45.892566598 (366h49m31.354659081s) [upload=322699]: Waiting for error group managing segments...
Oct 8 20:20:46.357939721 (366h49m31.820032194s) [noise=true , address=45.91.116.14:4189 , piece_id=Z456ZXXUZTK6BMGE5NA6FXAKKXQ5ZK6IJIETBXRITK6JMMGY6PVA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.549331821 (366h49m32.011424304s) [noise=true , address=45.91.116.11:4119 , piece_id=G7GVVGAUGMVC3MKTWGJZ536VQYXY6QSBC6C6PHVFHDCJBOXEIVAQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.605152004 (366h49m32.067244477s) [noise=true , address=45.91.116.8:4048 , piece_id=TRBGMNHKJFPYFZVUJ7N2ER7UKEUTEPFFEVGY2CDBTSHPPRV7UPLQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.757137313 (366h49m32.219229796s) [noise=true , address=45.91.116.7:4011 , piece_id=WD32BIXN3OCL6XV574UBQ4K2LGMF2NIFPXLJ36KIIVGJN7QNQWEA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.762592642 (366h49m32.224685125s) [noise=true , address=77.90.31.15:4698 , piece_id=AWSLS5P7AKW7TPJADENMT7M3UGUKIVS4JLENKZ2CCGKRUE4RZ2BA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.930160945 (366h49m32.392253418s) [noise=true , address=77.90.31.21:4877 , piece_id=J2J3XOBCQQRCHZV3Y4GEIOFO75MWMQQRDOHQF37HDSD6XN5SZVIA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.936545282 (366h49m32.398637765s) [noise=true , address=77.90.31.15:4697 , piece_id=QSX7CJ472UUJIWE6A5FX4WJDJ4EL74ERUTEALV4T2XFYTRV5V76A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:46.997082336 (366h49m32.459174819s) [noise=true , address=77.90.31.21:4879 , piece_id=DLLMF3W5E2BEZJ2XRKFTRUTE2Q5WT423EW2MFT32RPX4GLWS57IA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:20:47.112429610 (366h49m32.574522093s) [noise=true , address=77.90.31.16:4735 , piece_id=PMCJL5VFREMILH4IOC5MICGQTC34S7R6FWCVLHICQTJTVLLNBF5A, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:09.596227994 (366h49m55.058320507s) [noise=true , address=145.79.227.15:4874 , piece_id=7GFSKKHOOR5XVUL3V47X6FUJVHOEPOV5K5N52RIUCDMVQDEKUUAA, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:12.919226884 (366h49m58.381319367s) [noise=true , address=145.79.227.17:4972 , piece_id=XIOPE5PUKBFIKUG3GKV7YKYFGJ5U2NR4WW3XYMO2W5GWI7TDFO2Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: <nil>
Oct 8 20:21:13.012452124 (366h49m58.474544617s) [noise=true , address=145.79.227.7:4295 , piece_id=M5AW4B4ZPQKQ2U5GRKS6447OGBYMLOCY6YSDOUPFTEIKI6IVYPFQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39348->145.79.227.7:4295: use of closed network connection
Oct 8 20:21:13.012487522 (366h49m58.474579995s) [noise=true , address=145.79.227.6:4259 , piece_id=2LXND2EJC63PE5AJGGGJPB276RW7YTOEGIY7O5BM22IVZWOT2IRQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42694->145.79.227.6:4259: use of closed network connection
Oct 8 20:21:13.012606899 (366h49m58.474699382s) [noise=true , address=145.79.227.2:4115 , piece_id=ZNFSZYLZJGOUKD6TJONORMC3KZTOYHXXORJ5D7LYVPYRWT2E5O7Q, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45376->145.79.227.2:4115: use of closed network connection
Oct 8 20:21:13.012619673 (366h49m58.474712147s) [noise=true , address=145.79.227.16:4911 , piece_id=X74CKTFG3HTXQ4CZB7V3AHUMQSBIAQEQ3FDWFUSNUBNAWBNQR5XQ, seg_pos={0 0}, upload=322699]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48018->145.79.227.16:4911: use of closed network connection
Oct 8 20:21:13.012651835 (366h49m58.474744308s) [seg_pos={0 0}, upload=322699]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil>
Oct 8 20:21:13.257982894 (366h49m58.720075367s) [upload=322699]: Done uploading segments...
Oct 8 20:21:13.258018953 (366h49m58.720111426s) [upload=322699]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11)
Duration: 5.926388866s
core
uplink
us-select-1
|