Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
6.185763615s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:34:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.185881009s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.787615181s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:33:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.787647702s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
5.625562914s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:33:24 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.625591178s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
6.126966423s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:33:01 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.126995679s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.699614104s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:32:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.699641246s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.802961958s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:32:13 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.802987585s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.735835815s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:31:40 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.735864529s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.731421959s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:30:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.73145338s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
5.844652423s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:30:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.844685437s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.305940286s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:29:31 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.305982797s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
6.131963379s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:28:30 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.131981604s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.81389277s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:27:37 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.813920372s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
5.644617029s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:27:21 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.644667506s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.718535416s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:27:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.718584349s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
5.728424193s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:26:10 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.728459189s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.152774264s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:25:53 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.152803829s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.726580104s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:25:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.726613036s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.723388012s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:24:20 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.723414111s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
5.722419298s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:23:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.722453062s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
5.651151329s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:23:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.651189673s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
5.809342838s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:22:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.809368616s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
5.325056183s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:21:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.325085538s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.667397359s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:20:42 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.667425883s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
5.791011467s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:20:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.791045623s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.813678333s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:18:54 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.813696246s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.763266864s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:17:52 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.763346315s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.341294453s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:16:46 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.341316305s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
5.653260491s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:16:00 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.653297262s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.750663252s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:15:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.75069332s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
5.879260501s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:14:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.879292501s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
5.741412554s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:13:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.741443823s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.239931842s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:12:09 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.239961669s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.441578843s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:11:16 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.441607607s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
6.268366826s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:10:05 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.268401052s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.122973744s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:09:17 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.122999402s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
6.189611394s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:08:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.189661059s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
7.537470759s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:07:06 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 7.53749788s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.750627867s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:06:35 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 5.750658546s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
8.290913545s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:06:03 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 8.29094785s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
6.532205945s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:05:45 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 6.53223405s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
9.736324978s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:04:56 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 9.736353051s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
28.435890639s trace:
https://jaeger.dev.storj.io/trace/d24e282cae355a8
FAIL (2025-11-29 23:04:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: eestream: quiescence
======LOGS======
63204 (495h0m13.918633665s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367386489 (495h0m13.91885694s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.367586149 (495h0m13.919056611s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.368368469 (495h0m13.91983893s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.370168877 (495h0m13.921639338s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.372146263 (495h0m13.923616714s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.373978502 (495h0m13.925448963s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.374415854 (495h0m13.925886596s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.375566746 (495h0m13.927037207s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:15.379743138 (495h0m13.931213599s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Uploading piece...
Nov 29 23:04:16.193964208 (495h0m14.745434669s) [upload=435893]: Next returned nil segment
Nov 29 23:04:16.193980790 (495h0m14.745451241s) [upload=435893]: Waiting for error group managing segments...
Nov 29 23:04:16.588328531 (495h0m15.139798982s) [noise=true , address=185.167.204.205:4022 , piece_id=KQGAPJ7IJQDN6N4LEIQTD43THU2KPWLMYSFUYKPBBE23K5MPTXYA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.588450043 (495h0m15.139920504s) [noise=true , address=185.167.204.206:4061 , piece_id=R6OFY6MJVY5ENXOZZWHC2CKS73QNXJIWDRBKXTYJRXNOOQYTKDOA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.591055714 (495h0m15.142526175s) [noise=true , address=185.167.204.205:4015 , piece_id=BH4FVVQEWVY5OP3GHPVVHYXV7YXXL7VBEKVBNICNQIUMZOQMBSAA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.595531226 (495h0m15.147001677s) [noise=true , address=185.167.204.205:4005 , piece_id=YFK5HBNTOTKRSUS4XRZ3YRPAI4ZEK3DKBLXSIG56DZPOPCTXPOKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.596866599 (495h0m15.14833706s) [noise=true , address=185.167.204.205:4033 , piece_id=YP3HYJ5S3HAPYPPQ5X4BBKI7LERBP7LOVOKZHWMR5GTZC6QMWE3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600090748 (495h0m15.151561209s) [noise=true , address=185.167.204.206:4072 , piece_id=IQ4AAJG4NXO7I2JOJTSSDWF5KXZGLIN6H4BZVUOMVWRSWEIPQHRA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.600437999 (495h0m15.15190846s) [noise=true , address=185.167.204.205:4021 , piece_id=GISPTI7P22CQBEETXZ7S4K4RKLTOCAGNKPEQNJK4SA6OZ3LSIE7Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.606498990 (495h0m15.157969441s) [noise=true , address=185.167.204.206:4055 , piece_id=B3BO65PLZH6WDE7YB4PNBRIZPKDWA7E2LIUQZL5WZB7MYJFXSIVQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607132467 (495h0m15.158602918s) [noise=true , address=185.167.204.206:4056 , piece_id=BS26JRUAU2EQAW4FY3IRCVHLH6W34RM55ZRD5HZQFEZ72MTOK7UA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.607358097 (495h0m15.158828558s) [noise=true , address=185.167.204.206:4037 , piece_id=ROTQBBJ5FEBBTGFOYIS7DL4YWH4WEW3QAOS354UH7J5N2EBLIAAQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.608394009 (495h0m15.15986447s) [noise=true , address=185.167.204.205:4023 , piece_id=SMMT4AZUTPV5ILN5DNN6ORRJNNZUVHISOVW5NZYP5KL266V5J3PQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.609654600 (495h0m15.161125061s) [noise=true , address=185.167.204.205:4014 , piece_id=PETYMCW3E3SIYMDAG3HZRA4ITZAWFPEZVS6PQ2UEF22SLTKOIBWQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610336178 (495h0m15.161806639s) [noise=true , address=185.167.204.206:4041 , piece_id=UVQAYYJ6JBH2B7F3GF5PJBWNVKKWWPYCNAQ7EPAEVMOQ6QBFJ34A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.610894210 (495h0m15.162364671s) [noise=true , address=185.167.204.206:4064 , piece_id=36FVLSJCAL36EI6RP2ZB2TRZ3DSBB2K2WIVYWTD6E2SJK2VMOR4Q, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.612907824 (495h0m15.164378285s) [noise=true , address=185.167.204.206:4040 , piece_id=JI3FSPBHYZXSTJSXXADXEZKO24UGVPYITM64Z7ORHFSPQQS7PE7A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.613654436 (495h0m15.165124887s) [noise=true , address=185.167.204.205:4020 , piece_id=P6XJ6EVG4HEEC4MNEFKCTOZOX55EJIMTTL4RSWPC5MTS77MVOA4A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.614271882 (495h0m15.165742343s) [noise=true , address=185.167.204.206:4062 , piece_id=H3SUGOYJRWVMUSGMLIG5GFW5PHWTVVM6M6XJOUJ6OSY7VAL2UM5A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615166645 (495h0m15.166637096s) [noise=true , address=185.167.204.206:4038 , piece_id=ZU7CEJCYELXC7FKN2QPJ5IAZP6EFPPV4J4J66RH3KCDJV23275RA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.615702125 (495h0m15.167172586s) [noise=true , address=185.167.204.206:4071 , piece_id=IYOTAVZNF2NHM4EENETKXNZEDDOSQ2S5JM6Y6WU2C42XMEUEZSPQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.622348872 (495h0m15.173819323s) [noise=true , address=185.167.204.205:4036 , piece_id=NDGICHF2D2AAG4BTTH5HWCZL4S3WZS5MFEBJDTDVIOP7GGZ6EJ6A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.623554238 (495h0m15.175024699s) [noise=true , address=185.167.204.206:4045 , piece_id=YSXBKASXLB4PWVBCXYKKZKFLWYDSCNLRLAZWTDLXK5RGM5JLIBQA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.625468773 (495h0m15.176939234s) [noise=true , address=185.167.204.205:4011 , piece_id=LZEFBH7J46P5EMRPFCWE4YCLP3MP6N77IHQN3CW6CQS57O75MYKQ, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:16.633697702 (495h0m15.185168163s) [noise=true , address=185.167.204.206:4065 , piece_id=7BVM7XCIK6L3Y5PZHS4IWBI7SCQDGPZ6NHRMZ7BKKPPNEAZASP3A, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:17.952847920 (495h0m16.504318381s) [noise=true , address=89.149.201.147:4007 , piece_id=FYFK5AZQPRL5X34B5LH53Z2ONLLDNRHNBJ2Q4AU2APNX23BU5TBA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.168814250 (495h0m16.720284711s) [noise=true , address=81.171.14.70:4032 , piece_id=4MIVIZSF3NYEAJE26RP2WWY4YVDVBJCVGWEGVGE6DZUXBKNJUFWA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905289666 (495h0m17.456760137s) [noise=true , address=89.149.201.147:4016 , piece_id=GUK3X3E3WEMODEBKDKN2LBLLSANLIGR7TKZH2SCEIH54H5SYROMA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: <nil>
Nov 29 23:04:18.905315666 (495h0m17.456786137s) [seg_pos={0 0}, upload=435893]: Segment reached optimal threshold of 26 pieces.
Nov 29 23:04:18.905784339 (495h0m17.45725481s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng): protocol: expected piece hash; context canceled; EOF
Nov 29 23:04:18.905833892 (495h0m17.457304364s) [noise=true , address=81.171.14.70:4031 , piece_id=2D6GBD5N3VMLYPWEGIA7HBYBUETVEFPVADJ6N2DVVGXXOFDETJDA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906246979 (495h0m17.45771745s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12RTn3BuxasVSPYmjqgyhqKn1t6fAtq9EBtUufXqcLq48My9g2b): protocol: read tcp 188.166.225.170:44936->81.171.14.70:4026: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Nov 29 23:04:18.906260154 (495h0m17.457730615s) [noise=true , address=81.171.14.70:4026 , piece_id=LDE2MVOSRJ7PSKIQXMT2JXSTTZQSWE2N35BWZRA2I6NB77YU7CEA, seg_pos={0 0}, upload=435893]: Long tail cancellation due to optimal threshold
Nov 29 23:04:18.906282096 (495h0m17.457752567s) [seg_pos={0 0}, upload=435893]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Nov 29 23:04:18.906373140 (495h0m17.457843601s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Nov 29 23:04:18.906380694 (495h0m17.457851165s) [upload=435893]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Nov 29 23:04:19.169071817 (495h0m17.720542278s) [upload=435893]: Done uploading segments...
Nov 29 23:04:19.169097004 (495h0m17.720567465s) [upload=435893]: Upload finished. err: <nil>
Duration: 28.435955643s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.748006657s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:06:08 UTC)
orbiter warning: unexpected EOF
Duration: 5.748026355s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
6.076866345s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:05:48 UTC)
orbiter warning: unexpected EOF
Duration: 6.076899548s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
5.820924347s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:05:21 UTC)
orbiter warning: unexpected EOF
Duration: 5.820938102s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
37.009458799s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:05:15 UTC)
orbiter warning: unexpected EOF
Duration: 37.00948592s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.587091874s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:03:50 UTC)
orbiter warning: unexpected EOF
Duration: 5.587139194s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
5.748958579s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:02:57 UTC)
orbiter warning: unexpected EOF
Duration: 5.748989267s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
5.889063755s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:02:06 UTC)
orbiter warning: unexpected EOF
Duration: 5.88908173s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
6.078486976s trace:
https://jaeger.dev.storj.io/trace/370356ccc2b8dde6
FAIL (2025-11-29 21:01:28 UTC)
orbiter warning: unexpected EOF
Duration: 6.078521162s
core
uplink
us-select-1
us1-placement-20
|