Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......X....................................................] duration:
3.556194157s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:09:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.556224155s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
5.242792425s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:09:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.242814909s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
467.786528ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:09:56 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.811817ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [........X...........X...................X...................] duration:
5.878492677s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:09:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.87852114s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.898861274s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:09:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.898891081s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
5.64945866s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:09:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.649492023s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
466.537259ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:09:25 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 466.557057ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........X...................................................] duration:
3.52860874s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:09:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.528643937s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
2.035872994s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:09:16 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.035902149s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
5.967587211s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:09:01 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.967618952s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........X..................................................] duration:
3.575300945s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:09:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.575320853s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.........X...........X...................X..................] duration:
5.735021343s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:08:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.735046411s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
6.25647321s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:08:43 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.256500982s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........X.................................................] duration:
3.5247903s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:08:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.52481705s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
2.026590752s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:08:17 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.02662661s
core
uplink
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
473.113222ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:08:13 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 473.142116ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
5.993085759s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:08:10 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.993112468s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...........X................................................] duration:
3.364393673s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:08:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.364426876s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..........X...........X...................X.................] duration:
5.922454077s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:07:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.922522316s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................] duration:
466.766946ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:07:58 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 466.792615ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
5.757810963s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:07:48 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.757843245s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
4.865918063s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:07:44 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.865952217s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
30.181431396s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:07:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 30.181463586s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
2.000269228s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:07:27 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.00029614s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [............X...............................................] duration:
3.639323574s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:07:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.639350485s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................] duration:
467.846332ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:07:06 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.870629ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.....................................................X.....] duration:
5.627125013s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:06:46 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.62716005s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...........X...........X...................X................] duration:
5.650869096s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:06:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.650921536s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
467.214498ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:06:35 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.242373ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.....................................................X....] duration:
5.759444085s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:06:29 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.759488941s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
5.275082135s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:06:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.275110829s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
2.019817262s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:06:24 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.019851187s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [............X...........X...................X...............] duration:
6.031419895s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:06:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.03144346s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.............X..............................................] duration:
3.421356983s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:06:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.421381701s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
4.574779036s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:06:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.57480729s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
2.039495375s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:06:06 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.039523239s
core
uplink
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
468.234601ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:05:50 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 468.258898ms
core
satellite
eu1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
1.952843315s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:05:49 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 1.952866919s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
5.670086157s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:05:42 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.670117226s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.....................................................X...] duration:
5.840994662s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:05:32 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.841022665s
core
uplink
us1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
25.408389631s trace:
https://jaeger.dev.storj.io/trace/3d28e5ace62e0e01
FAIL (2025-12-08 20:05:27 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
OK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.293932973 (76h13m38.165016008s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294413788 (76h13m38.165496823s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.294875628 (76h13m38.165958663s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.301577459 (76h13m38.172660524s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.304616707 (76h13m38.175699752s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.307486942 (76h13m38.178569977s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.311116633 (76h13m38.182199668s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.314576141 (76h13m38.185659187s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.318186856 (76h13m38.189269911s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:38.320152179 (76h13m38.191235214s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Uploading piece...
Dec 8 19:42:39.134825843 (76h13m39.005908878s) [upload=66666]: Next returned nil segment
Dec 8 19:42:39.134842836 (76h13m39.005925881s) [upload=66666]: Waiting for error group managing segments...
Dec 8 19:42:39.519924371 (76h13m39.391007406s) [noise=true , address=185.167.204.205:4029 , piece_id=BAEIL5V3IHBHWYB77MIDBNGX356OXUXAXU4KKP7I7I5SYMC6QKEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520407791 (76h13m39.391490826s) [noise=true , address=185.167.204.206:4058 , piece_id=DIDUUF4SQ2XFG5N3T3PAFYDKGF3YDYXYAJHZ4BFFWBIB5J47ZPOA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.520919345 (76h13m39.39200239s) [noise=true , address=185.167.204.206:4059 , piece_id=ZXOANAVMA33NV37MSQUDEU2QVRWQQ3AHAGOQ4J4YOYWDYWLWNVIQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.524350188 (76h13m39.395433233s) [noise=true , address=185.167.204.205:4023 , piece_id=JQGURY62AF5IEARL2K3C3XGW6S57UZO6QR345JGVB2IN46OZNXTQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.528022881 (76h13m39.399105916s) [noise=true , address=185.167.204.206:4060 , piece_id=IPMGIYN5R4UZSNXFUG7KDQL5D6LCJCPLLPJCCERJ52BT7ASMMIHQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.535814137 (76h13m39.406897182s) [noise=true , address=185.167.204.206:4051 , piece_id=DOMDNAWAWOV5NS5UFFMZWYRVKZFXIUEU7ZBTVT5SRKFQLQRGDZKQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541502700 (76h13m39.412585735s) [noise=true , address=185.167.204.206:4069 , piece_id=6NDEHGDMCACFBJEKOJDWDEGD3CSZDMSXEDQBOPBG5FVLQJMC4QYQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.541942957 (76h13m39.413025993s) [noise=true , address=185.167.204.206:4066 , piece_id=BONIZ563CJFAW7ADZ6FDZAEMT6CJESN7Z7BMTBCQCO2WE3INL2RQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.542708876 (76h13m39.413791961s) [noise=true , address=185.167.204.206:4046 , piece_id=KSPWZJAUZPXIBGHIWXTRXIBOPYF42ZBRWQL2XRCHSUIC35NCUFDQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.543549516 (76h13m39.414632602s) [noise=true , address=185.167.204.205:4016 , piece_id=3E3GOK2NZ4HXKQHC5MTTRY2YLP7YYS7SBJUU7M365LU56VLVAUJQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.546598362 (76h13m39.417681397s) [noise=true , address=185.167.204.206:4045 , piece_id=Z4IGGM74BYWYDOC5M6C6NK574ZOBBFZ4M7DFTR7LUDR7DKD2LUAA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.548463354 (76h13m39.419546399s) [noise=true , address=185.167.204.205:4022 , piece_id=QVHNV4M6BJIYQSLFFM6PHZ4NO2Z4F7S2UFL3UNOIBVTI6PBLX7UQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.549008301 (76h13m39.420091346s) [noise=true , address=185.167.204.206:4047 , piece_id=TWTCTSIARGE2AOCLMF6YO7RLOL7YSZN5D7PDCNAXOX26S6KWC4ZA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552460625 (76h13m39.42354367s) [noise=true , address=185.167.204.206:4040 , piece_id=VS4ZLJJWZUPUBAIH6CC4W7VHOLMZWYKDWWAOOAIPCG7WP3VJEHYA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.552710752 (76h13m39.423793797s) [noise=true , address=185.167.204.206:4057 , piece_id=X5Y7PZGFIQ5KE6PHQN5YOHUM773U6NKOGF5DPWPG76YXRPMWLMHA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.553051219 (76h13m39.424134264s) [noise=true , address=185.167.204.205:4018 , piece_id=4SCEDL7KSFFSBPWAEGKETTTWFDGN5NID4ZCFRVJ6QZ2JTXVHS2XQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:39.557215800 (76h13m39.428298835s) [noise=true , address=185.167.204.205:4014 , piece_id=CJZ6K3EONWY3ZBAEFQ2GBBZ5PSYVHOMCP2EFKKERUG4QW27CSN2Q, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.092656704 (76h13m39.963739739s) [noise=true , address=81.171.14.70:4044 , piece_id=NPX7C4PJNRXJFFFWS72F4NTXNWBECFDTIVSNIQEV5N4UAYKV3ZCQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.133122829 (76h13m40.004205874s) [noise=true , address=81.171.14.70:4030 , piece_id=JFXGK7KJLPI5LJXGCADL5GYQZVKOSJMTO2VYHGIS4NWU44HNORVQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.135113711 (76h13m40.006196746s) [noise=true , address=81.171.14.70:4039 , piece_id=MXY6PIXD5CBG2ZDQK3QJZBIXUTJURV57UOZLC7QQCAYZGTJ2UHEQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.157260182 (76h13m40.028343217s) [noise=true , address=81.171.14.70:4048 , piece_id=D2N5VBUAMWTK3754BEKSK2OYRIFXNYUCDMOZIG3NJTZG5URN4ULA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.158303388 (76h13m40.029386423s) [noise=true , address=81.171.14.70:4036 , piece_id=ET46ZX2EMELT4OASYNU4WR4DNITLS2VA4FP422OVQXASV6TWU3MA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.539176087 (76h13m40.410259122s) [noise=true , address=81.171.14.70:4040 , piece_id=YIXCA6XG3O2DB3QM2R2O7IEWKH2JIA6TBOB46EA5SOAYIEBXKVKA, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.547733031 (76h13m40.418816076s) [noise=true , address=81.171.14.70:4033 , piece_id=IBR7PYBXDE3L5UELKVXP5WDVZI72VRJVRWJ7MDOHXHQB24ZUVV3A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.566289897 (76h13m40.437372942s) [noise=true , address=81.171.14.70:4047 , piece_id=Q2RTM6SANOOKVSRN2XGXFJ6LN5GSFR24IAGSU6JO3AOMZYHJZ4DQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628077722 (76h13m40.499160757s) [noise=true , address=81.171.14.70:4042 , piece_id=7UBOZ3TM6U6KLXKV7VOKZNEY6ABWB5VDKGDV6OSZSAXHEKLRNZ4A, seg_pos={0 0}, upload=66666]: Done uploading piece. err: <nil>
Dec 8 19:42:40.628096548 (76h13m40.499179583s) [seg_pos={0 0}, upload=66666]: Segment reached optimal threshold of 26 pieces.
Dec 8 19:42:40.628363045 (76h13m40.499446111s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1foiaFSv2TTCkBBCnVYJ6xRbAJ2YYkhnKKmtZJY31ZAzFMTWQQ): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628373044 (76h13m40.499456079s) [noise=true , address=89.149.201.147:4017 , piece_id=QLG2MVERXCEFCXBBJBWYZARUNKS6VPOUUSXSUJOGX2G5CCTH6BGQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628734754 (76h13m40.499817789s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sNB8ZUfm8aR6jWBvWvVx6w7zZgGvbw4kuUCqMHipRWmU8o5R8): protocol: expected piece hash; context canceled; EOF
Dec 8 19:42:40.628744101 (76h13m40.499827146s) [noise=true , address=89.149.201.147:4001 , piece_id=D65FANROOLI47T6542V4YMZ5HJ6YG5BG6IS2H27KYIR3P4WNH2YQ, seg_pos={0 0}, upload=66666]: Long tail cancellation due to optimal threshold
Dec 8 19:42:40.628767596 (76h13m40.499850641s) [seg_pos={0 0}, upload=66666]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil>
Dec 8 19:42:40.628886502 (76h13m40.499969547s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Dec 8 19:42:40.628893004 (76h13m40.49997606s) [upload=66666]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Dec 8 19:48:22.295722109 (76h19m22.166805154s) [upload=66666]: Done uploading segments...
Dec 8 19:48:22.295853299 (76h19m22.166936354s) [upload=66666]: Upload finished. err: metaclient: context deadline exceeded
Duration: 25.408418687s
core
uplink
eu-select-1
eu1-placement-15
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [.............X...........X...................X..............] duration:
5.930055547s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:05:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.930083851s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
472.121153ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:05:17 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 472.170686ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..............X.............................................] duration:
3.449271337s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:05:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.449301142s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
2.000423484s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:05:13 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.000450014s
core
uplink
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.....................................................X..] duration:
5.833947152s trace:
https://jaeger.dev.storj.io/trace/4905938924e243c2
FAIL (2025-12-08 20:04:29 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 5.83397744s
core
uplink
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [..............X...........X...................X.............] duration:
5.738617215s trace:
https://jaeger.dev.storj.io/trace/6b0ac9718f78e692
FAIL (2025-12-08 20:04:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.738647473s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
2.016929253s trace:
https://jaeger.dev.storj.io/trace/21f9fb0e12c4fe9
FAIL (2025-12-08 20:04:21 UTC)
orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded
Duration: 2.016957275s
core
uplink
us1
|
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
467.124327ms remote:
"tcp:35.207.121.91:443" target:
https://eu1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/5e91473084188cf7
FAIL (2025-12-08 20:04:21 UTC)
orbiter warning: Get "https://eu1.storj.io/login": EOF
Duration: 467.152671ms
core
satellite
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [...............X............................................] duration:
3.445903113s trace:
https://jaeger.dev.storj.io/trace/624d2b5b2aee8843
FAIL (2025-12-08 20:04:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.445944852s
core
uplink
us-select-1
us1-placement-20
|