Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......X...................X................................] duration:
10.58056275s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:12:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.58057893s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....................................................X.X.....] duration:
3.400027488s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:12:41 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.400057977s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........X...................X...............................] duration:
12.558332797s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:12:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 12.558364036s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....................................................X.X....] duration:
5.000806247s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:11:43 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000829883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........X...................X..............................] duration:
5.875980545s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:11:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 5.876008578s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......................................................X.X...] duration:
3.534251425s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:10:40 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.534277484s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........X...................X.............................] duration:
6.222467276s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:10:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 6.222499888s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........X...................X............................] duration:
13.162106079s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:09:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 13.16213306s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......................................................X.X..] duration:
3.44938754s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:09:35 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.449426886s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............X...................X...........................] duration:
9.490055957s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:09:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 9.490087517s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........................................................X.X.] duration:
3.503937952s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:09:13 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.50395323s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............X...................X..........................] duration:
10.720142282s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:09:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.720172209s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..............X...................X.........................] duration:
5.222940189s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:08:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 5.222968994s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................................................X.X] duration:
5.001690439s trace:
https://jaeger.dev.storj.io/trace/39a0c2c3b39b2fe6
FAIL (2025-12-05 23:08:28 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001709847s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................................X.] duration:
3.443655244s trace:
https://jaeger.dev.storj.io/trace/63b730c2569a9678
FAIL (2025-12-05 23:07:44 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.443684411s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...............X...................X........................] duration:
12.02207659s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:07:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 12.022114471s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [................X...................X.......................] duration:
10.341355724s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:07:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.341384107s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........................................................X] duration:
5.001001587s trace:
https://jaeger.dev.storj.io/trace/63b730c2569a9678
FAIL (2025-12-05 23:06:54 UTC)
orbiter warning: operation error S3: DeleteObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.001022798s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.................X...................X......................] duration:
11.923732592s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:05:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.923761276s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..................X...................X.....................] duration:
13.840349587s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:05:35 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 13.840377109s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................X...................X....................] duration:
11.084815514s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:04:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.084841432s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....................X...................X...................] duration:
9.921213802s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:03:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 9.921240874s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................X...................X..................] duration:
11.769710608s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:03:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.769734934s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................X...................X.................] duration:
9.737597773s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:02:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 9.737625616s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................X...................X................] duration:
10.039508221s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:02:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.039536216s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [........................X...................X...............] duration:
10.138042441s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:01:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.138073289s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.........................X...................X..............] duration:
7.528681321s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:00:39 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 7.528707931s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..........................X...................X.............] duration:
11.499199447s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 23:00:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.499236738s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...........................X...................X............] duration:
10.345344754s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:59:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.345451307s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [............................X...................X...........] duration:
11.820203572s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:58:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.820229121s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.............................X...................X..........] duration:
6.213494188s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:58:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 6.213522724s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..............................X...................X.........] duration:
9.784464981s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:57:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 9.784482514s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...............................X...................X........] duration:
10.703421691s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:57:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.703449332s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [................................X...................X.......] duration:
10.502449762s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:56:52 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.502481832s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.................................X...................X......] duration:
10.660380863s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:56:04 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.660412093s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [..................................X...................X.....] duration:
10.649102074s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:54:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.64912729s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [...................................X...................X....] duration:
10.235801709s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:53:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.235833471s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [....................................X...................X...] duration:
10.009552677s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:52:47 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.009577585s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X...................X..] duration:
11.585189925s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:52:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.585218008s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [......................................X...................X.] duration:
32.209693857s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:51:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 32.209721069s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X...................X] duration:
1m0.16848004s trace:
https://jaeger.dev.storj.io/trace/2876f02a87e51eb3
FAIL (2025-12-05 22:51:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.734852567 (7h21m11.605935612s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.735096091 (7h21m11.606179136s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736258334 (7h21m11.607341369s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.736982783 (7h21m11.608065828s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.737666465 (7h21m11.60874951s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.738491607 (7h21m11.609574682s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.740068118 (7h21m11.611151153s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.746266501 (7h21m11.617349556s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.747818487 (7h21m11.618901522s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:11.748788444 (7h21m11.619871479s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Uploading piece...
Dec 5 22:50:12.561676428 (7h21m12.432759473s) [upload=6494]: Next returned nil segment
Dec 5 22:50:12.561692619 (7h21m12.432775664s) [upload=6494]: Waiting for error group managing segments...
Dec 5 22:50:12.944616244 (7h21m12.815699279s) [noise=true , address=185.167.204.206:4069 , piece_id=RXFXSLJ5INV2W7MKCYQYT5IA3YIDH7ERNQPFTNEWQI5IEPDZALCA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955382786 (7h21m12.826465831s) [noise=true , address=185.167.204.206:4060 , piece_id=3323KAJYX2P56USPDDJOGV5IJ6XMDEXD7WAT7NEPGGQPYIZV3BGA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.955638413 (7h21m12.826721458s) [noise=true , address=185.167.204.205:4009 , piece_id=REGQOE2WUJQ4ZXEKSYW5OQ5FT7KKIZBYTRONPAT4734YTEDBKYMQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.957909026 (7h21m12.828992071s) [noise=true , address=185.167.204.206:4072 , piece_id=ILLXRD3E7TWZWYBQCDZPBWQQGOPP7HBTMWOT6VYLWNM6FBSTLHDQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.961902330 (7h21m12.832985375s) [noise=true , address=185.167.204.205:4024 , piece_id=SM7MB4VE6SEBJU3QXIYSPJLHXWV4VBKPAMEVODQXPYXALU3X3T2A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.963965348 (7h21m12.835048393s) [noise=true , address=185.167.204.205:4013 , piece_id=3T63IAYX24K765HFO2F3SRW22W75ZPTULQRQP6CB4S2SMMRIS24Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.964543690 (7h21m12.835626735s) [noise=true , address=185.167.204.205:4034 , piece_id=U57F7B7MFII3VTIY2HYRMCRIMPBFCZDP7RZOAALIUM6K37I4NL3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966780339 (7h21m12.837863384s) [noise=true , address=185.167.204.206:4047 , piece_id=ETKC3Y2FFSRWELMBOCE25K2BUMABJ7MD235UIYJQH4VVGOKPJSRA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.966864499 (7h21m12.837947544s) [noise=true , address=185.167.204.206:4063 , piece_id=KCKPYEWU3EDQ36ZPUBV43NMT44HDKCN72FNTF53AS3O7SB7NOT3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969165922 (7h21m12.840248967s) [noise=true , address=185.167.204.205:4008 , piece_id=XUWCRDU5JSEKS3J54X5P6SYFVHGXRLLZGO472WAHJ2X44P4TTVXA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.969330344 (7h21m12.840413389s) [noise=true , address=185.167.204.206:4056 , piece_id=FCYXLPGNU7FBCABSX42ELRAEYNBOSM7ZAGCHMSROPLOQXQ3MZZNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971373886 (7h21m12.842456931s) [noise=true , address=185.167.204.206:4051 , piece_id=EWQ2ZYPLD76GDYJMVH3G2FQTHMN2JVR4346UUTFOGCSRQJ3H2MYA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.971912952 (7h21m12.842995997s) [noise=true , address=185.167.204.205:4006 , piece_id=2WP2HW7BSWVJ5P3RA2MAWRNRMQL27MBXNPYQIPC3ZPWAKRHUJJ3Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.972337321 (7h21m12.843420366s) [noise=true , address=185.167.204.206:4071 , piece_id=DGJYAEKS5DKZ5Z2NZNA5KMBHWRPLIYZTLZ2NP6RU5FIUBIRI5KBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.974738563 (7h21m12.845821608s) [noise=true , address=185.167.204.206:4039 , piece_id=U2GBOGJL7TXPJCD2OWFXTTFYKZ2Y7HGCOFEPNWGCHDSX7A4ZC73A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.975508028 (7h21m12.846591063s) [noise=true , address=185.167.204.205:4026 , piece_id=W5CKOD4VLEZR2VJRYQ7QWXX5W3RVAENRVT52LIGNLTVIM2VVXHNQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976083343 (7h21m12.847166388s) [noise=true , address=185.167.204.205:4014 , piece_id=YNYAX4HU236GQVIMZ5Y7DAFC62I5DRXQBVKHDVKQVDOQWRLQ3O5Q, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.976566353 (7h21m12.847649398s) [noise=true , address=185.167.204.206:4064 , piece_id=CG63SO632OQD3LDM7IUZ7D27JZBIR2UKDBSYMTATYTRVLK6IJVBQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.977893460 (7h21m12.848976505s) [noise=true , address=185.167.204.205:4027 , piece_id=5KUASPNP4BVKLYEMYQX6JUZD3KXNQF3Z6HJDOAQEIPP7M2JRPPEA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:12.982665797 (7h21m12.853748852s) [noise=true , address=185.167.204.206:4044 , piece_id=6K3MI6Z7BFLCQLO42N4YABKCZYL6KNBM3D7N7K2RCLZCCUNL56XA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.115610157 (7h21m13.986693192s) [noise=true , address=89.149.201.147:4005 , piece_id=RSSIE6CVK5WQ7HLDV3CIDWQDLB6ZZCT3KONIYQ63IDYRFS246CCQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.669771991 (7h21m14.540855036s) [noise=true , address=89.149.201.147:4004 , piece_id=KALCWL4U6UWVJIUBULSMQTLXNJUD4JCDI2ZKKFA75KVBME2PSILQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:50:14.759279209 (7h21m14.630362244s) [noise=true , address=89.149.201.147:4014 , piece_id=YUOWKE42KZJRGLIHJY37S7JVHN5DV57RBNPXSD2V2WVPYZC2CSDA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: <nil>
Dec 5 22:51:11.086071807 (7h22m10.957154842s) [noise=true , address=81.171.14.70:4045 , piece_id=OLH3P6N6BP62FC65QHBCW5P2T5TPFJ5TWT5F6OLMZDOWNZW6ASAQ, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: read tcp 188.166.225.170:40260->81.171.14.70:4045: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086122805 (7h22m10.95720584s) [noise=true , address=81.171.14.70:4039 , piece_id=EDKBZPMGNDVYTGVLWZQQV2EXTSYBUO56M2P4NVERVFQLAIDOZALA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1qsAHeMX7hadREKwHeJGrLGBCVjc223xFbZ53943A2RvJnSVuC, address:81.171.14.70:4039): protocol: context deadline exceeded; context deadline exceeded
Dec 5 22:51:11.086240619 (7h22m10.957323654s) [noise=true , address=81.171.14.70:4025 , piece_id=AH3GJAIQQB2BIEAPAIFBO7TDGUEHN6SPEHFUO5TL2C4E6LRAHY7A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:1ruq1atBi7bkJYK7Kqv63js28sW75imA1SwTytzLneZqRRSGFU, address:81.171.14.70:4025): protocol: read tcp 188.166.225.170:32908->81.171.14.70:4025: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086260216 (7h22m10.957343251s) [noise=true , address=81.171.14.70:4036 , piece_id=B3CVBLYFBDXRPBCGD3AUBBV4H3OLVDTG5QBXR3GQX5BYOM7IRDTA, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12TcRxUCpNQVo3KLtKPK1zc6NXR4AdRCvqH4gtxticRH5MrKhVq, address:81.171.14.70:4036): protocol: read tcp 188.166.225.170:50656->81.171.14.70:4036: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086434508 (7h22m10.957517543s) [noise=true , address=81.171.14.70:4030 , piece_id=YKHOD4TGQRXUCX6Q5GLWVXX42C6ZVQ3Z6WGFT67ETYGVHZFPOG5A, seg_pos={0 0}, upload=6494]: Done uploading piece. err: ecclient: upload failed (node:12ZoRwa8wLw66Pjumy1ybp6wrZ79RrSU55uYZpwaY3xi95iSYde, address:81.171.14.70:4030): protocol: read tcp 188.166.225.170:39920->81.171.14.70:4030: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
Dec 5 22:51:11.086463593 (7h22m10.957546628s) [seg_pos={0 0}, upload=6494]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:51:11.252000935 (7h22m11.12308398s) [upload=6494]: Done uploading segments...
Dec 5 22:51:11.252034539 (7h22m11.123117574s) [upload=6494]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 1m0.168508254s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
10.724552324s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:49:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 10.724580109s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
13.428031247s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:48:55 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 13.428060043s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
9.746116445s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:48:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 9.746143397s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
16.034044842s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:47:28 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 16.034071714s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
19.206870501s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:46:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 19.206897352s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
11.619066202s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:46:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 11.619113542s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
13.228593191s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:45:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 13.228622546s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
13.427242184s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:44:23 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 13.427269565s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
16.807635028s trace:
https://jaeger.dev.storj.io/trace/5113fc9f394095e4
FAIL (2025-12-05 22:43:29 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 26 but got 23)
======LOGS======
=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160276663 (7h4m31.031359698s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160360482 (7h4m31.031443517s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160631759 (7h4m31.031714804s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160698446 (7h4m31.031781491s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.160973750 (7h4m31.032056796s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161022493 (7h4m31.032105528s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161322024 (7h4m31.032405059s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161370235 (7h4m31.03245328s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161697369 (7h4m31.032780414s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.161725872 (7h4m31.032808907s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.162022557 (7h4m31.033105602s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.164057994 (7h4m31.035141039s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Uploading piece...
Dec 5 22:33:31.986942610 (7h4m31.858025645s) [upload=6254]: Next returned nil segment
Dec 5 22:33:31.986954283 (7h4m31.858037328s) [upload=6254]: Waiting for error group managing segments...
Dec 5 22:33:32.380676103 (7h4m32.251759138s) [noise=true , address=185.167.204.206:4070 , piece_id=OE44IOFLK7IFKAZOO3HISXH5ZOOLF73TYNEPMPFIQH5JCC2L3RBA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.382415826 (7h4m32.253498861s) [noise=true , address=185.167.204.206:4039 , piece_id=WBEHGKIKCYB5RAEUACSRVTWTEIXLKLTI3HIQY27SVTT254TT7RHA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384037635 (7h4m32.25512067s) [noise=true , address=185.167.204.206:4064 , piece_id=KLNKJDVEWOYWEOZ66M32QUX3D3OU7PUX4WGGFCFHAYO3STALGMFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.384653487 (7h4m32.255736532s) [noise=true , address=185.167.204.206:4046 , piece_id=2P5POPEWEOP4L64TYCR7O63RRADR2CW7LEOEW5AUZ5VCXZAGWBIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.385162877 (7h4m32.256245912s) [noise=true , address=185.167.204.206:4065 , piece_id=FCNVGF6Q5NKQZVTGRSNVFCBFA73WXVUALZQNJ5HQD4FAX753J3KA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.387589528 (7h4m32.258672563s) [noise=true , address=185.167.204.206:4067 , piece_id=RIQIYNYO6AUI3VT2Y5MX2ZZKUMAFCJJWXTOM3GBEIURNJZT23XCA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.394822311 (7h4m32.265905346s) [noise=true , address=185.167.204.206:4040 , piece_id=N5QEDZ6MZZS6VA3QYZFCWUKYLJEWEAK524TDRFPY46WVFOLKYIFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402472498 (7h4m32.273555533s) [noise=true , address=185.167.204.205:4019 , piece_id=4ESJVIGXPGRZ2SJDNS6B666B22L2P2Z62YX5UPLM7EJS7EKMY7YA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402552240 (7h4m32.273635275s) [noise=true , address=185.167.204.205:4004 , piece_id=JQVFE2LB7WMESQBC2YUXUDJNPVCRGCI7LPRGW34UIFEQOBAIVSPQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.402895223 (7h4m32.273978258s) [noise=true , address=185.167.204.205:4034 , piece_id=5LDD5KA7STKHXIYTKP2CAIZTKLCQXHNDHL2OL2FDEGRZTOCBMM5A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403006595 (7h4m32.27408964s) [noise=true , address=185.167.204.205:4028 , piece_id=BGAMHAV3T5CX3V2XGNXM3SR2DGSHXM2BQERNQVXDUEO2GVAFDR2Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403253545 (7h4m32.27433659s) [noise=true , address=185.167.204.205:4026 , piece_id=LITMCMQBWCJZJDAAQ7YIHEV3ESSO6ALZJVMSNWU53UGEYNYTPMYA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:32.403602720 (7h4m32.274685755s) [noise=true , address=185.167.204.205:4003 , piece_id=4KZYOUFELESN5ULWARAWBSCRGD5E3M5CA3TZECJ6FTWY5SNP7WBQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.586947310 (7h4m33.458030356s) [noise=true , address=89.149.201.147:4019 , piece_id=3N6DEPUBVFUHRMTMCKEETJSUIXJ2WVOVPBHCGEDQVSU5TQWLNNFA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.634434465 (7h4m33.5055175s) [noise=true , address=89.149.201.147:4014 , piece_id=SKSPVE5KWOGF27ZC322E2GHUPWX5D6GOAZSXB366LEOKWLIFXTJQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640408711 (7h4m33.511491756s) [noise=true , address=89.149.201.147:4016 , piece_id=KWSSSMSSH6F2Q7BOIY6VHFW27N2GMVFPTINK3HVNQN3KNOVJ5WEQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.640879898 (7h4m33.511962943s) [noise=true , address=89.149.201.147:4006 , piece_id=QC4AK5U2NHQSCNRP6AGZ2YVBZ7EZDD4TOB422SAMG5MLMQ6YXU7A, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.644746661 (7h4m33.515829706s) [noise=true , address=89.149.201.147:4010 , piece_id=TTB4URUUGAOULJ4WSN7YHW3YFD4JBKL6SPXN26MNFHJ3KRLTXCPA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:33:33.651448683 (7h4m33.522531718s) [noise=true , address=89.149.201.147:4005 , piece_id=6HXVDTI6SO5QDD3RNNYPSCMCMAEMJJ4WWQWNYWDZDI5AKKGLN4QA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.345912763 (7h5m2.216995808s) [noise=true , address=81.171.14.70:4030 , piece_id=UYXO2WLBJ7LYX53NLOI4XVDASSZ7D4E47AAC4VDXPYISEFZ2KBZQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:02.477063068 (7h5m2.348146113s) [noise=true , address=81.171.14.70:4036 , piece_id=BWZPZRGR75KKTSVDSQYK6DJCEGYNOS3USJBM2JZ35DY7FAVQ5E7Q, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:05.306834854 (7h5m5.177917899s) [noise=true , address=81.171.14.70:4032 , piece_id=FPBWHQVPB4WJXTIYKSHY2BV2SGMW5EM4DPC2YJWBJ5JUDGKK7QDQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:29.554821351 (7h5m29.425904396s) [noise=true , address=81.171.14.70:4029 , piece_id=22J2NEB5J32BEJRQIMXW5QX2KKCW2PXPZZBUAFJICB6E7AAV7NIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: <nil>
Dec 5 22:34:30.507846490 (7h5m30.378929535s) [noise=true , address=81.171.14.70:4042 , piece_id=WDK5AQC34LBAXFL2WHKRRXDXLHE5GHYMNITP4VPJAQJ2LUSKJAIQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12vBozgWFrZfUEQAwsqBt6CDMW47rVjZaDfvKTsCdPrvX8swkhD, address:81.171.14.70:4042): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507930640 (7h5m30.379013675s) [noise=true , address=81.171.14.70:4027 , piece_id=WCMXRR72JR6Y472VQ6PRGN3VC3JMGQFU26AO54VEYHUAY66QXHMA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:122cYvgTxSvC2aEXnBkRk2o8ARTBXj9gqPoa12vW8hogSN6kpA4, address:81.171.14.70:4027): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.507985104 (7h5m30.379068149s) [noise=true , address=81.171.14.70:4031 , piece_id=57PDRQR4RRUZLZSTPPICNTHWJHZHDC5AK7LQFAP6EL6YWSR52HGA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12drdYnnWsuMyFo6LkcewAGGWnorRCAhxPpNqK41LvGy2zkBvng, address:81.171.14.70:4031): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508060036 (7h5m30.379143072s) [noise=true , address=81.171.14.70:4044 , piece_id=MF4KRDDWO7C57VB63MHSKR4IVQEGJVGDB3KF37MWC4JUNDEFYKSA, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:1MUVGecysnTSFcJoHUL7QMWb3h6dtXgUd7FbA672fQCyEWhePr, address:81.171.14.70:4044): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508148434 (7h5m30.379231479s) [noise=true , address=81.171.14.70:4045 , piece_id=HWN5CFMGP3NYIFG6WTAECRIMFDHDPTO2MCAAPN7VBEF7KY6OZYSQ, seg_pos={0 0}, upload=6254]: Done uploading piece. err: ecclient: upload failed (node:12WCMeEEkwoqbPeyENfJt6nahQz2h3yW7st1TtvoY6KDKZqUGmS, address:81.171.14.70:4045): protocol: expected piece hash; context deadline exceeded; EOF
Dec 5 22:34:30.508177119 (7h5m30.379260154s) [seg_pos={0 0}, upload=6254]: Done waiting for segment. successful: 23 optimal: 26 errs: <nil>
Dec 5 22:34:30.677594522 (7h5m30.548677557s) [upload=6254]: Done uploading segments...
Dec 5 22:34:30.677640259 (7h5m30.548723304s) [upload=6254]: Upload finished. err: failed to upload enough pieces (needed at least 26 but got 23)
Duration: 16.807661257s
core
uplink
eu-select-1
eu1-placement-15
|