Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
5.92590444s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:24:21 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.92593071s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.......................X.X.....X............................] duration:
5.882430855s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:24:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.882458838s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
6.227462683s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:24:15 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.227492019s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...............................X......X.....................] duration:
15.557575227s remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:24:15 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 15.557604833s
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..X.............X.......XX........X.........................] duration:
3.52602584s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:24:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.526053653s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [........................X.X.....X...........................] duration:
5.924971113s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:23:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.925171104s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
6.005805907s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:23:44 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.005835164s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........................X.X.....X..........................] duration:
5.926577109s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:23:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.926616714s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
5.867255106s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:23:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.867301154s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [................................X......X....................] duration:
467.840497ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:22:59 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.866485ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [...X.............X.......XX........X........................] duration:
3.462533097s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:22:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.462605887s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..........................X.X.....X.........................] duration:
5.874707793s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:22:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.874732159s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
6.02544591s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:22:43 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.025469265s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [....X.............X.......XX........X.......................] duration:
3.409169353s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:22:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.409188221s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
6.236480982s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:22:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.236512723s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.................................X......X...................] duration:
480.489597ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:21:57 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 480.520216ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
6.232663234s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:21:54 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.2326927s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...........................X.X.....X........................] duration:
5.854345607s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:21:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.85437438s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.065563127s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:21:41 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.065601731s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.....X.............X.......XX........X......................] duration:
3.510293523s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:21:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.510346564s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [............................X.X.....X.......................] duration:
5.793797719s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:21:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.793828528s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [..................................X......X..................] duration:
477.611088ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:21:08 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 477.644681ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
5.82275733s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:20:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.822807878s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [...................................X......X.................] duration:
467.113933ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:20:49 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.216127ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [......X.............X.......XX........X.....................] duration:
3.530264613s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:20:46 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.530414578s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.............................X.X.....X......................] duration:
5.932580937s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:20:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.932615083s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
5.737611936s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:20:25 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.737638206s
core
uplink
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [....................................X......X................] duration:
483.445866ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:20:08 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 483.489558ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
6.167960823s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:20:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.168059821s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
6.670457046s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:19:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.670487032s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..............................X.X.....X.....................] duration:
5.838999494s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:19:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.839029432s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X......X...............] duration:
8.492504512s remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:19:49 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 8.492535411s
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.......X.............X.......XX........X....................] duration:
3.417183031s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:19:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.417210113s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...............................X.X.....X....................] duration:
5.735845103s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:19:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.735874678s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
6.322411172s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:19:14 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.3224464s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [........X.............X.......XX........X...................] duration:
3.480506563s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:19:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.480527062s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
6.065109352s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:18:53 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 6.065137326s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [.........X.............X.......XX........X..................] duration:
3.402783896s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:18:44 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.402814364s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [......................................X......X..............] duration:
466.075977ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:18:39 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 466.112206ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [................................X.X.....X...................] duration:
5.633891935s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:18:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.63392126s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
5.878562632s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:18:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.878585895s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
5.888082864s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:18:26 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.888112431s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [..........X.............X.......XX........X.................] duration:
3.484858326s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:18:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.484889816s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.756267176s trace:
https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea
FAIL (2025-12-09 02:18:01 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece...
Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment
Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments...
Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil>
Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces.
Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF
Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold
Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments...
Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.756301432s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X......X.............] duration:
476.420732ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:17:57 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 476.444987ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.90176946s trace:
https://jaeger.dev.storj.io/trace/40019ff87d088ff7
FAIL (2025-12-09 02:17:31 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.901799357s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.................................X.X.....X..................] duration:
6.047684678s trace:
https://jaeger.dev.storj.io/trace/5280cd334ed3ed6b
FAIL (2025-12-09 02:17:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.047714413s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [........................................X......X............] duration:
467.163398ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:17:24 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.190999ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
5 out of 60 is failed (threshold=6) [...........X.............X.......XX........X................] duration:
3.428793843s trace:
https://jaeger.dev.storj.io/trace/dbaa73a4151be2e
FAIL (2025-12-09 02:17:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.428837655s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X......X...........] duration:
468.473202ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b
FAIL (2025-12-09 02:17:09 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 468.504863ms
core
satellite
eu1
|