Orbiter History

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.X......X...................................................] duration: 468.759373ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:44:27 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 468.803777ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.910659016s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:44:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.910687891s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [X........X........................................X...X.....] duration: 3.503646214s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:43:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503671523s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [X.X.....X..........................................X........] duration: 5.686710442s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:43:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.686740278s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 7.658559388s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:43:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.658582612s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [..X......X..................................................] duration: 472.047826ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:43:43 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 472.076659ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.238002592s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:43:41 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.238037327s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [XX........X........................................X...X....] duration: 3.518387707s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:43:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.51841488s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.X.X.....X..........................................X.......] duration: 5.849915189s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:43:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.850111775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.260426446s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:43:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.260476512s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [..X.X.....X..........................................X......] duration: 5.793541274s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:43:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.79357037s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [...X......X.................................................] duration: 477.478987ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:43:15 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 477.535706ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.169750098s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:43:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.169853443s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.284803983s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:43:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.28482816s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.900971995s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:42:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.901245765s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.210057338s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:42:51 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.210103386s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [....X......X................................................] duration: 473.79953ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:42:49 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 473.827614ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [.XX........X........................................X...X...] duration: 3.779570377s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:42:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.779598491s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.956020019s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:42:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.95605769s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.262825887s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:42:32 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.262856836s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.125312481s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:42:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.125343569s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [...X.X.....X..........................................X.....] duration: 5.773065996s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:42:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.77308358s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.868025967s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:42:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.868057267s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [..XX........X........................................X...X..] duration: 3.461657805s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:42:02 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.461689434s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [....X.X.....X..........................................X....] duration: 5.701917719s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:41:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.701963778s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.....X......X...............................................] duration: 476.814262ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:41:49 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.848376ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.996352908s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:41:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.996380862s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 4.978461111s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:41:40 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.978489666s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.....X.X.....X..........................................X...] duration: 5.663730534s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:41:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.663762625s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.011107075s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:41:22 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.011265217s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 4.910425952s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:41:09 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 4.910547081s core uplink eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [...XX........X........................................X...X.] duration: 3.686729134s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:41:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.686758971s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.030389178s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:40:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.030414307s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [......X.X.....X..........................................X..] duration: 5.727942158s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:40:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.72796904s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.384225182s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:40:43 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.384252824s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 5 out of 60 is failed (threshold=6) [....XX........X........................................X...X] duration: 5.001897248s trace: https://jaeger.dev.storj.io/trace/5a3e8bc7e1c5fe0 FAIL (2025-12-09 02:40:42 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001925383s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.777395682s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:40:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.77742034s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [......X......X..............................................] duration: 470.063157ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:40:36 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 470.158177ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.243412973s trace: https://jaeger.dev.storj.io/trace/78baf37c16400f4c FAIL (2025-12-09 02:40:07 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.243476556s core uplink eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.965751001s trace: https://jaeger.dev.storj.io/trace/40019ff87d088ff7 FAIL (2025-12-09 02:40:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.96578632s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [.......X......X.............................................] duration: 685.395443ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:40:03 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 685.426954ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.28075032s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:40:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.280782541s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.807479895s trace: https://jaeger.dev.storj.io/trace/5b7efc901355db5e FAIL (2025-12-09 02:39:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581295090 (83h6m52.452378125s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.581634846 (83h6m52.452717891s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.582888624 (83h6m52.453971659s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583267836 (83h6m52.454350871s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583663720 (83h6m52.454746766s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.583984842 (83h6m52.455067887s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584326653 (83h6m52.455409698s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.584654216 (83h6m52.455737251s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585020815 (83h6m52.45610386s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.585435474 (83h6m52.456518519s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586429306 (83h6m52.457512351s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.586955358 (83h6m52.458038403s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588208955 (83h6m52.459292s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.588553080 (83h6m52.459636115s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:52.589933539 (83h6m52.461016574s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Uploading piece... Dec 9 02:35:54.045728988 (83h6m53.916812023s) [upload=72598]: Next returned nil segment Dec 9 02:35:54.045753754 (83h6m53.916836799s) [upload=72598]: Waiting for error group managing segments... Dec 9 02:35:54.429970583 (83h6m54.301053618s) [noise=true , address=145.79.227.8:4334 , piece_id=DMWK5WVDG7VZWAWBV6UILCHWA57J2C2D7DHMXTJBM4UWJTF3OWPA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.444386584 (83h6m54.315469629s) [noise=true , address=145.79.227.5:4233 , piece_id=D4V4LNQ7QOP3OQDTWDBUXHGHAS7IYPXARV3AYCPKGWZTEO6OTICA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.469193360 (83h6m54.340276395s) [noise=true , address=145.79.227.38:4996 , piece_id=JUFG4APIQSTWSS4MQ6JUBKLTEOHVV64YU6ZTPLMWMRF346HBO6GQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.472137816 (83h6m54.343220862s) [noise=true , address=145.79.227.22:4540 , piece_id=2ATNCIUNDNDD57MSDWU356YSJY4UMRZSQFLJGXZQQWQSVO7E7OJA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.484648119 (83h6m54.355731204s) [noise=true , address=145.79.227.51:4436 , piece_id=NKWQM7OVKVCQCEFF6WSY3E653UMT5RBOWDX6WSQAD7NJ6VK5RKMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.489783557 (83h6m54.360866602s) [noise=true , address=145.79.227.26:4656 , piece_id=IC4AUOX2QE44UDPFEWAOFDY4W25RX24DKU4WPFKSQWM74EJNM75A, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.596419028 (83h6m54.467502073s) [noise=true , address=77.90.31.21:4887 , piece_id=IXSCAD7EUGK3ZZ4H6R3F5N2WF7MOI4DBTAVVFFIBUT2NKRJCBULA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.598540137 (83h6m54.469623172s) [noise=true , address=77.90.31.15:4714 , piece_id=LP7HCHJ3TXO5OJWGDJFV5RMKASVDMLBB5G6NKTZKLOBHHVMG5T2Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.602190628 (83h6m54.473273674s) [noise=true , address=45.91.116.8:4046 , piece_id=SCSARQHG5I6JB4365O5MPJRCJAA6UKHLR6XHY6EQYOH5CRMR4ZQQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.604555752 (83h6m54.475638797s) [noise=true , address=45.91.116.13:4164 , piece_id=XV3OXMXPJTFXBFMND7EMW3NUPW4P2ZQVL63CTXPDCMMZDTNOZMMQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.623163304 (83h6m54.494246349s) [noise=true , address=77.90.31.19:4875 , piece_id=27ACR5MATWJSBAQBMN6MSFA4LJODOW2FPG4WDLBJASIRVBOGI73Q, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.630125902 (83h6m54.501208947s) [noise=true , address=77.90.31.14:4691 , piece_id=CHKKJVWU5IFXLODN2RDB3SILIWGBZGJ7RLPNEDWYFIFIUGZIDKPQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.644849627 (83h6m54.515932672s) [noise=true , address=45.91.116.10:4104 , piece_id=TGQVBFKOVCHHS76B6HQBX6RNCNHV7XBE3JW2CUCTEO54VV7OVLWQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661873844 (83h6m54.532956889s) [noise=true , address=45.91.116.14:4214 , piece_id=U5UMWUHBQELI7M23VDC2GJNGYG676B4LT7Z7Z5DHGOYRSW3JOSHQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: <nil> Dec 9 02:35:54.661936824 (83h6m54.533019869s) [seg_pos={0 0}, upload=72598]: Segment reached optimal threshold of 14 pieces. Dec 9 02:35:54.662405798 (83h6m54.533488843s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Dfm6iHQbMhj1YWLu3UboTCtZ2hq87aiYUgH7Kig3pDUrPeFj4): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662415215 (83h6m54.53349826s) [noise=true , address=45.91.116.9:4272 , piece_id=4DUWR3IDCLYLEGOK2DC7GILHO3UW24H34PG36PVB6UGYDOPUHCVA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662471652 (83h6m54.533554697s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7): protocol: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35816->64.49.13.169:4706: use of closed network connection Dec 9 02:35:54.662517330 (83h6m54.533600375s) [noise=true , address=64.49.13.169:4706 , piece_id=IPMPTX7W572TPX34J5I6VGLSQMB44MYQISE6WYTUN4G2BJXMDNNA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662678426 (83h6m54.533761461s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1V8iMvJLcTddWZihWQBQLLCyMTwbD2h7ruDUjMEtHD7EHtNVEK): protocol: write tcp 188.166.225.170:42636->77.90.31.16:4766: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:35:54.662687404 (83h6m54.533770449s) [noise=true , address=77.90.31.16:4766 , piece_id=5Z4SH4GYB3QB2CBHF3A2NMUSH5QI575GKN7B7EFKL6NBF7X7M3ZQ, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662891562 (83h6m54.533974597s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XjJpLBEZm5daHcBUmUyC7QpLVhu9Vm9ex9RYcYJWEuLGk527F): protocol: expected piece hash; context canceled; EOF Dec 9 02:35:54.662901071 (83h6m54.533984106s) [noise=true , address=77.90.31.13:4640 , piece_id=C3QWIVLOZSJDB2MWMEBEPPLVPF4H3HHBCJH7ZWAF64BWDQOI65GA, seg_pos={0 0}, upload=72598]: Long tail cancellation due to optimal threshold Dec 9 02:35:54.662923623 (83h6m54.534006658s) [seg_pos={0 0}, upload=72598]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 9 02:35:54.663067688 (83h6m54.534150723s) [upload=72598]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:35:54.663073940 (83h6m54.534156985s) [upload=72598]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:38:38.394271693 (83h9m38.265354758s) [upload=72598]: Done uploading segments... Dec 9 02:38:38.394386772 (83h9m38.265469807s) [upload=72598]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.807502938s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 4 out of 60 is failed (threshold=6) [.....XX........X........................................X...] duration: 3.559657202s trace: https://jaeger.dev.storj.io/trace/2ebaa2d702006440 FAIL (2025-12-09 02:39:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.559765157s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.................X.....] duration: 5.783558076s trace: https://jaeger.dev.storj.io/trace/215a258615585c9a FAIL (2025-12-09 02:39:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 02:36:33.367818130 (83h7m33.238901175s) [upload=72607]: Starting upload Dec 9 02:36:33.367959239 (83h7m33.239042284s) [upload=72607]: Done starting upload Dec 9 02:36:33.368038239 (83h7m33.239121274s) [upload=72607]: Uploading segments... Dec 9 02:36:33.368256765 (83h7m33.23933981s) [upload=72607]: Got next segment. Inline: false Dec 9 02:36:33.368313543 (83h7m33.239396588s) [upload=72607]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 02:36:33.368321098 (83h7m33.239404153s) [upload=72607]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 02:36:33.612196015 (83h7m33.48327905s) [seg_pos={0 0}, upload=72607]: Begin upload segment... Dec 9 02:36:33.612249908 (83h7m33.483332943s) [seg_pos={0 0}, upload=72607]: Done begin upload segment. Dec 9 02:36:33.612277811 (83h7m33.483360856s) [noise=true , address=77.90.31.15:4698 , piece_id=BPN4QMAI6U2EU3RGZ45V37A2VBNKHJVZR5GJ6A77Q52QCYINJZQQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.612790757 (83h7m33.483873802s) [noise=true , address=145.79.227.20:4437 , piece_id=5P6TCISX2VNJ35JV33UVYWEUAV7D5PZN4MS4W7GSRZRRZ4LOUSEQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.612954920 (83h7m33.484037965s) [noise=true , address=145.79.227.33:4871 , piece_id=6N3JMFBKG3QMF6KIV5BIC5Z3QAI7FZ5ALL5J2V2ODUAYKKKYWJOQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613299877 (83h7m33.484382922s) [noise=true , address=145.79.227.23:4546 , piece_id=YMDL4F2QYIY2OCWI3P46FSAMWYTVQXVXIFZF6WT4C44REOLSESLA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613628723 (83h7m33.484711768s) [noise=true , address=145.79.227.43:4157 , piece_id=CPR63S5GOTNQAU5UIWZPKIRGPOHKE5VJIY22MNXJRRDAOF4E354A, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613938673 (83h7m33.485021718s) [noise=true , address=145.79.227.9:4366 , piece_id=4EMHO4GLNH73AOE7243VGYAGEJR7FR7GZQ6XL2LZ25YJU2746AUQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614272509 (83h7m33.485355554s) [noise=true , address=145.79.227.11:4732 , piece_id=IZEWLKSKSK5GN7V7LUCRSUYBNXZR4DIJZJPHI4LXLHV7KIV7FSOQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614924269 (83h7m33.486007304s) [noise=true , address=145.79.227.28:4727 , piece_id=RJXEG4WHX4PZYTBWCZV6KL6O46D62GUUKAJJHSHCMIXYO2KEND6A, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614941944 (83h7m33.486024989s) [noise=true , address=145.79.227.41:4086 , piece_id=S6CEQVRFEGR42PXS6OZSMQ6XPA57MHJEAI6L2ZPYU67DGRGQ4YWA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615209043 (83h7m33.486292078s) [noise=true , address=145.79.227.32:4836 , piece_id=PQLQTDAMCLHNHKUGEPLBJQDOPGRPCGF2W366EEUPWUBJWKV4UFPA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615334100 (83h7m33.486417145s) [noise=true , address=145.79.227.39:4015 , piece_id=A7AGQIA6MJR6M4A5QL3HPLA6ITTAYXSNNDLI6XCC5KXNVRITYPUA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615551794 (83h7m33.486634829s) [noise=true , address=145.79.227.20:4439 , piece_id=GU2KFL4DK7PYWJKILSGPCYHUI2CNYEOMIEXJY7SGFYDNXWUDUMJA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615914726 (83h7m33.486997771s) [noise=true , address=145.79.227.16:4910 , piece_id=MOZZ27FIQRWYHA5UJCOH3GKMKVICAF5WKJA5SBYYCMR6TZ2P4YUQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.616325487 (83h7m33.487408522s) [noise=true , address=145.79.227.34:4907 , piece_id=HO3VDRVKFFFWL2I7TC2V5AKWFYIORAVQOGJI7I237QPDDSHVQ2WQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.616886486 (83h7m33.487969522s) [noise=true , address=77.90.31.19:4841 , piece_id=MX2HJM56PK4IWSIDPLRFYZJC6SJIEKKZJ7D2Y3PMLEQEMXTTV4MA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.617302177 (83h7m33.488385222s) [noise=true , address=145.79.227.40:4050 , piece_id=OBWXSVL6BOHZJN7N4X6HP356AMAA6KI4PM66G3RN5TXF323QMGXQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:35.216821640 (83h7m35.087904675s) [upload=72607]: Next returned nil segment Dec 9 02:36:35.216835827 (83h7m35.087918872s) [upload=72607]: Waiting for error group managing segments... Dec 9 02:36:35.598648069 (83h7m35.469731114s) [noise=true , address=145.79.227.9:4366 , piece_id=4EMHO4GLNH73AOE7243VGYAGEJR7FR7GZQ6XL2LZ25YJU2746AUQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.599953394 (83h7m35.471036439s) [noise=true , address=145.79.227.33:4871 , piece_id=6N3JMFBKG3QMF6KIV5BIC5Z3QAI7FZ5ALL5J2V2ODUAYKKKYWJOQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.600927218 (83h7m35.472010264s) [noise=true , address=145.79.227.43:4157 , piece_id=CPR63S5GOTNQAU5UIWZPKIRGPOHKE5VJIY22MNXJRRDAOF4E354A, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.602614362 (83h7m35.473697407s) [noise=true , address=145.79.227.20:4437 , piece_id=5P6TCISX2VNJ35JV33UVYWEUAV7D5PZN4MS4W7GSRZRRZ4LOUSEQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.607613370 (83h7m35.478696425s) [noise=true , address=145.79.227.20:4439 , piece_id=GU2KFL4DK7PYWJKILSGPCYHUI2CNYEOMIEXJY7SGFYDNXWUDUMJA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.608421910 (83h7m35.479504945s) [noise=true , address=145.79.227.32:4836 , piece_id=PQLQTDAMCLHNHKUGEPLBJQDOPGRPCGF2W366EEUPWUBJWKV4UFPA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.610690881 (83h7m35.481773916s) [noise=true , address=145.79.227.40:4050 , piece_id=OBWXSVL6BOHZJN7N4X6HP356AMAA6KI4PM66G3RN5TXF323QMGXQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.621471418 (83h7m35.492554453s) [noise=true , address=145.79.227.41:4086 , piece_id=S6CEQVRFEGR42PXS6OZSMQ6XPA57MHJEAI6L2ZPYU67DGRGQ4YWA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.625350385 (83h7m35.49643343s) [noise=true , address=145.79.227.16:4910 , piece_id=MOZZ27FIQRWYHA5UJCOH3GKMKVICAF5WKJA5SBYYCMR6TZ2P4YUQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.626939180 (83h7m35.498022225s) [noise=true , address=145.79.227.11:4732 , piece_id=IZEWLKSKSK5GN7V7LUCRSUYBNXZR4DIJZJPHI4LXLHV7KIV7FSOQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.630536180 (83h7m35.501619225s) [noise=true , address=145.79.227.34:4907 , piece_id=HO3VDRVKFFFWL2I7TC2V5AKWFYIORAVQOGJI7I237QPDDSHVQ2WQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.632515828 (83h7m35.503598873s) [noise=true , address=145.79.227.39:4015 , piece_id=A7AGQIA6MJR6M4A5QL3HPLA6ITTAYXSNNDLI6XCC5KXNVRITYPUA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:37.566200474 (83h7m37.437283519s) [noise=true , address=77.90.31.19:4841 , piece_id=MX2HJM56PK4IWSIDPLRFYZJC6SJIEKKZJ7D2Y3PMLEQEMXTTV4MA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 9 02:37:02.685025066 (83h8m2.556108112s) [noise=true , address=77.90.31.15:4698 , piece_id=BPN4QMAI6U2EU3RGZ45V37A2VBNKHJVZR5GJ6A77Q52QCYINJZQQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Dec 9 02:37:02.685361857 (83h8m2.556444892s) [noise=true , address=145.79.227.23:4546 , piece_id=YMDL4F2QYIY2OCWI3P46FSAMWYTVQXVXIFZF6WT4C44REOLSESLA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: read tcp 188.166.225.170:44422->145.79.227.23:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:37:02.686002387 (83h8m2.557085432s) [noise=true , address=145.79.227.28:4727 , piece_id=RJXEG4WHX4PZYTBWCZV6KL6O46D62GUUKAJJHSHCMIXYO2KEND6A, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:36574->145.79.227.28:4727: use of closed network connection; context deadline exceeded Dec 9 02:37:02.686061210 (83h8m2.557144255s) [seg_pos={0 0}, upload=72607]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 02:37:02.933792900 (83h8m2.804875945s) [upload=72607]: Done uploading segments... Dec 9 02:37:02.933926595 (83h8m2.80500964s) [upload=72607]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.783599777s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [.......X.X.....X..........................................X.] duration: 5.735199133s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:39:40 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.735215465s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 4 out of 60 is failed (threshold=3) [........X.X.....X..........................................X] duration: 1m0.001178421s trace: https://jaeger.dev.storj.io/trace/341908f57c2faae7 FAIL (2025-12-09 02:39:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001198651s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 2 out of 60 is failed (threshold=3) [........X......X............................................] duration: 465.978036ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/4e3d2b60e3f4041b FAIL (2025-12-09 02:39:28 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 466.008816ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.208695023s trace: https://jaeger.dev.storj.io/trace/6f1ff10066b1d8ea FAIL (2025-12-09 02:39:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== rue , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223188835 (82h32m5.09427188s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.223808365 (82h32m5.09489141s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224324407 (82h32m5.095407452s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.224877841 (82h32m5.095960886s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.225476030 (82h32m5.096559075s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226119545 (82h32m5.09720258s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.226816121 (82h32m5.097899166s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.227549758 (82h32m5.098632813s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228376403 (82h32m5.099459448s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:05.228990242 (82h32m5.100073297s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Uploading piece... Dec 9 02:01:06.054905806 (82h32m5.925988851s) [upload=72112]: Next returned nil segment Dec 9 02:01:06.054930403 (82h32m5.926013448s) [upload=72112]: Waiting for error group managing segments... Dec 9 02:01:06.441184862 (82h32m6.312267897s) [noise=true , address=185.167.204.205:4004 , piece_id=6CC2A4V4TJPRJBN4IMJ7VGDVFYTJIVY6J3ZIPZWYL5TIBLGT744A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.444455871 (82h32m6.315538916s) [noise=true , address=185.167.204.205:4005 , piece_id=QUO2YNK2MJJ63CKWADK5K64GYKHDOSBPFHKYKQSKH2TJQUZTNEVA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.447822552 (82h32m6.318905597s) [noise=true , address=185.167.204.206:4066 , piece_id=SSGUSCBUP2YNFN7OMUZQIXJJMSHV6GDHQYMN475LJFQQ7SV2FV2A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.451954059 (82h32m6.323037104s) [noise=true , address=185.167.204.205:4008 , piece_id=54IT2P63UNQKCHG4HB7CX7UWJFWDLJXXPOGPNAEN36ZAC4UQ6VQQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454212810 (82h32m6.325295856s) [noise=true , address=185.167.204.205:4011 , piece_id=6JR2JQYKTRLCSBJDUWWSJ6CRYGWKWN3AW3KJCPXMYLYPUN6W2DRQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.454331416 (82h32m6.325414461s) [noise=true , address=185.167.204.206:4052 , piece_id=WY7PGT7EC5IUBTSHFEAMHXONCIKPFPRF3Y4YSL2IJNY2MOO6U7HA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.455138002 (82h32m6.326221047s) [noise=true , address=185.167.204.206:4050 , piece_id=YKLEM6U44VR3GM45KKKJF7JYOFA4YQE6P5UAZZEMXIA2LBZW5UKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.459504045 (82h32m6.33058709s) [noise=true , address=185.167.204.205:4013 , piece_id=7TBB5J7PYW23YFBOESEZ5Q6YOD3FKALVJMUQRMC5JSQ5JYKGDQSA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:06.465474574 (82h32m6.336557629s) [noise=true , address=185.167.204.206:4067 , piece_id=PK6GGQNT4DQ2ZMF2IEEO5X32IMA26ULKDN7HAAG4CWOC2UAUKMMA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.060075359 (82h32m6.931158404s) [noise=true , address=81.171.14.70:4034 , piece_id=DNHPDP5YCQLS5AN6OHWRWVOGOIGE53YB7CJVUHNPJK35AEM455GQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.111399558 (82h32m6.982482603s) [noise=true , address=81.171.14.70:4026 , piece_id=CRRJW4OCPQUSWT6BERGQJPYYQ4DHSZIBMIBCD22QWCGGHUISCFXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.133611052 (82h32m7.004694097s) [noise=true , address=81.171.14.70:4032 , piece_id=APFBMO7KRDDWH6WT3H4ZLCXBAAMZTZQYQBN3XX6JZDGOBAMSGVCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.149778297 (82h32m7.020861342s) [noise=true , address=81.171.14.70:4030 , piece_id=VOYOWF6Y5DZOJ6DLW6QFYUI6DF5YA7FI4VI57LHMNCZ2PZLRY4BQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.243693687 (82h32m7.114776732s) [noise=true , address=81.171.14.70:4040 , piece_id=RYDZ6DCJ3Z5YKJPIFMLR7EHPH55S7HABKAJCKGRGHF2TCRKEYHXQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.624315751 (82h32m7.495398796s) [noise=true , address=81.171.14.70:4046 , piece_id=RR4RRWZ6YO3EK62O7TE2QUC3ZJ762VGUTVXVVHPGMQRPWEFTB7VA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.682160867 (82h32m7.553243912s) [noise=true , address=89.149.201.147:4011 , piece_id=I77RBTAA7KPTBE27RPPPCNYU65WBBUORVCNYAJOV5OHOBGUFEJKA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.704962114 (82h32m7.576045159s) [noise=true , address=89.149.201.147:4003 , piece_id=KQDYYB3S7PMHBAZTSQDATYYBAUYHMHEQDI6PW4GM4HKXBK4XHIVQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.723657624 (82h32m7.594740669s) [noise=true , address=89.149.201.147:4008 , piece_id=JCJNV47ZC7LJPXSERGUXP2QRLHVPOAQNOPI25RQZTA52IYKRZK4Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.729409366 (82h32m7.600492411s) [noise=true , address=89.149.201.147:4002 , piece_id=YFUKN6FRXESUYMS6ZV53MCQTZV5UXYHFF72WS2XJWGF3K3I72GTQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.761878943 (82h32m7.632961988s) [noise=true , address=89.149.201.147:4004 , piece_id=NGEHOFVBOCBBCYIVGSYLMKTFQWTDJ2EOGKYEW4X2OIWYH5TVGX2Q, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.825863449 (82h32m7.696946494s) [noise=true , address=89.149.201.147:4005 , piece_id=K6WGQSQZ32ZSAD6KN5IWB66MJ3YP76IKULPEHTFSODYFBSESSU4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:07.866343990 (82h32m7.737427035s) [noise=true , address=89.149.201.147:4015 , piece_id=XBCH6ONGHITOZ2Z5GDPPKDV45TILCDFTOZ44OKMSZANCGVBGQ7ZQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.228780506 (82h32m8.099863541s) [noise=true , address=89.149.201.147:4017 , piece_id=5W66LIGVHUXIA4AVNKUNCOAJAP5TCYAKVFKCL6NDF2OVXNI62UMQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.230729618 (82h32m8.101812653s) [noise=true , address=89.149.201.147:4019 , piece_id=JTQ44O6RFCMIPK7LAEWXO2NTXAZAJK7XR3XAJGVQXLOLM7SFSBCA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.257428165 (82h32m8.12851121s) [noise=true , address=89.149.201.147:4020 , piece_id=T5RN3NMY35NXWM55YN3F7TWPPP7KXEER5R63W4KZV6Y2UNORJX4A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267970901 (82h32m8.139053946s) [noise=true , address=89.149.201.147:4006 , piece_id=M3VDXP334RID2FJK7ZYNUHJBRMS57EXIMVMLER25MVJWM4EHJ2WA, seg_pos={0 0}, upload=72112]: Done uploading piece. err: <nil> Dec 9 02:01:08.267990939 (82h32m8.139073974s) [seg_pos={0 0}, upload=72112]: Segment reached optimal threshold of 26 pieces. Dec 9 02:01:08.268172084 (82h32m8.139255129s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hJ5jT6S2d3DJe9DoVDjHMkpwSTpeyTMyNLaK8Pk8SWcUVRKnv): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268182163 (82h32m8.139265198s) [noise=true , address=89.149.201.147:4012 , piece_id=7UFT32WUAW2JUETM2Y4SFGMSPTNLVVH5FKPQJSK6S3DR6E64ITKQ, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268290639 (82h32m8.139373684s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1wgtphHKaTejC1qrJ98sDysBwtj2Zeqw2c5UFk1HMqfY8Buvr): protocol: expected piece hash; context canceled; EOF Dec 9 02:01:08.268299597 (82h32m8.139382642s) [noise=true , address=89.149.201.147:4014 , piece_id=PFXVAHNZ2Q6YC2MGNSUZMJIL5Q2O2HPDSONU3FD3E472H2G7AG5A, seg_pos={0 0}, upload=72112]: Long tail cancellation due to optimal threshold Dec 9 02:01:08.268340294 (82h32m8.139423329s) [seg_pos={0 0}, upload=72112]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 9 02:01:08.268435756 (82h32m8.139518802s) [upload=72112]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 02:01:08.268441557 (82h32m8.139524592s) [upload=72112]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 02:12:41.424380970 (82h43m41.295464025s) [upload=72112]: Done uploading segments... Dec 9 02:12:41.424504095 (82h43m41.29558713s) [upload=72112]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.208722246s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.................X....] duration: 5.771053373s trace: https://jaeger.dev.storj.io/trace/215a258615585c9a FAIL (2025-12-09 02:39:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Dec 9 02:36:33.367818130 (83h7m33.238901175s) [upload=72607]: Starting upload Dec 9 02:36:33.367959239 (83h7m33.239042284s) [upload=72607]: Done starting upload Dec 9 02:36:33.368038239 (83h7m33.239121274s) [upload=72607]: Uploading segments... Dec 9 02:36:33.368256765 (83h7m33.23933981s) [upload=72607]: Got next segment. Inline: false Dec 9 02:36:33.368313543 (83h7m33.239396588s) [upload=72607]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 02:36:33.368321098 (83h7m33.239404153s) [upload=72607]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 02:36:33.612196015 (83h7m33.48327905s) [seg_pos={0 0}, upload=72607]: Begin upload segment... Dec 9 02:36:33.612249908 (83h7m33.483332943s) [seg_pos={0 0}, upload=72607]: Done begin upload segment. Dec 9 02:36:33.612277811 (83h7m33.483360856s) [noise=true , address=77.90.31.15:4698 , piece_id=BPN4QMAI6U2EU3RGZ45V37A2VBNKHJVZR5GJ6A77Q52QCYINJZQQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.612790757 (83h7m33.483873802s) [noise=true , address=145.79.227.20:4437 , piece_id=5P6TCISX2VNJ35JV33UVYWEUAV7D5PZN4MS4W7GSRZRRZ4LOUSEQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.612954920 (83h7m33.484037965s) [noise=true , address=145.79.227.33:4871 , piece_id=6N3JMFBKG3QMF6KIV5BIC5Z3QAI7FZ5ALL5J2V2ODUAYKKKYWJOQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613299877 (83h7m33.484382922s) [noise=true , address=145.79.227.23:4546 , piece_id=YMDL4F2QYIY2OCWI3P46FSAMWYTVQXVXIFZF6WT4C44REOLSESLA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613628723 (83h7m33.484711768s) [noise=true , address=145.79.227.43:4157 , piece_id=CPR63S5GOTNQAU5UIWZPKIRGPOHKE5VJIY22MNXJRRDAOF4E354A, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.613938673 (83h7m33.485021718s) [noise=true , address=145.79.227.9:4366 , piece_id=4EMHO4GLNH73AOE7243VGYAGEJR7FR7GZQ6XL2LZ25YJU2746AUQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614272509 (83h7m33.485355554s) [noise=true , address=145.79.227.11:4732 , piece_id=IZEWLKSKSK5GN7V7LUCRSUYBNXZR4DIJZJPHI4LXLHV7KIV7FSOQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614924269 (83h7m33.486007304s) [noise=true , address=145.79.227.28:4727 , piece_id=RJXEG4WHX4PZYTBWCZV6KL6O46D62GUUKAJJHSHCMIXYO2KEND6A, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.614941944 (83h7m33.486024989s) [noise=true , address=145.79.227.41:4086 , piece_id=S6CEQVRFEGR42PXS6OZSMQ6XPA57MHJEAI6L2ZPYU67DGRGQ4YWA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615209043 (83h7m33.486292078s) [noise=true , address=145.79.227.32:4836 , piece_id=PQLQTDAMCLHNHKUGEPLBJQDOPGRPCGF2W366EEUPWUBJWKV4UFPA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615334100 (83h7m33.486417145s) [noise=true , address=145.79.227.39:4015 , piece_id=A7AGQIA6MJR6M4A5QL3HPLA6ITTAYXSNNDLI6XCC5KXNVRITYPUA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615551794 (83h7m33.486634829s) [noise=true , address=145.79.227.20:4439 , piece_id=GU2KFL4DK7PYWJKILSGPCYHUI2CNYEOMIEXJY7SGFYDNXWUDUMJA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.615914726 (83h7m33.486997771s) [noise=true , address=145.79.227.16:4910 , piece_id=MOZZ27FIQRWYHA5UJCOH3GKMKVICAF5WKJA5SBYYCMR6TZ2P4YUQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.616325487 (83h7m33.487408522s) [noise=true , address=145.79.227.34:4907 , piece_id=HO3VDRVKFFFWL2I7TC2V5AKWFYIORAVQOGJI7I237QPDDSHVQ2WQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.616886486 (83h7m33.487969522s) [noise=true , address=77.90.31.19:4841 , piece_id=MX2HJM56PK4IWSIDPLRFYZJC6SJIEKKZJ7D2Y3PMLEQEMXTTV4MA, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:33.617302177 (83h7m33.488385222s) [noise=true , address=145.79.227.40:4050 , piece_id=OBWXSVL6BOHZJN7N4X6HP356AMAA6KI4PM66G3RN5TXF323QMGXQ, seg_pos={0 0}, upload=72607]: Uploading piece... Dec 9 02:36:35.216821640 (83h7m35.087904675s) [upload=72607]: Next returned nil segment Dec 9 02:36:35.216835827 (83h7m35.087918872s) [upload=72607]: Waiting for error group managing segments... Dec 9 02:36:35.598648069 (83h7m35.469731114s) [noise=true , address=145.79.227.9:4366 , piece_id=4EMHO4GLNH73AOE7243VGYAGEJR7FR7GZQ6XL2LZ25YJU2746AUQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.599953394 (83h7m35.471036439s) [noise=true , address=145.79.227.33:4871 , piece_id=6N3JMFBKG3QMF6KIV5BIC5Z3QAI7FZ5ALL5J2V2ODUAYKKKYWJOQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.600927218 (83h7m35.472010264s) [noise=true , address=145.79.227.43:4157 , piece_id=CPR63S5GOTNQAU5UIWZPKIRGPOHKE5VJIY22MNXJRRDAOF4E354A, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.602614362 (83h7m35.473697407s) [noise=true , address=145.79.227.20:4437 , piece_id=5P6TCISX2VNJ35JV33UVYWEUAV7D5PZN4MS4W7GSRZRRZ4LOUSEQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.607613370 (83h7m35.478696425s) [noise=true , address=145.79.227.20:4439 , piece_id=GU2KFL4DK7PYWJKILSGPCYHUI2CNYEOMIEXJY7SGFYDNXWUDUMJA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.608421910 (83h7m35.479504945s) [noise=true , address=145.79.227.32:4836 , piece_id=PQLQTDAMCLHNHKUGEPLBJQDOPGRPCGF2W366EEUPWUBJWKV4UFPA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.610690881 (83h7m35.481773916s) [noise=true , address=145.79.227.40:4050 , piece_id=OBWXSVL6BOHZJN7N4X6HP356AMAA6KI4PM66G3RN5TXF323QMGXQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.621471418 (83h7m35.492554453s) [noise=true , address=145.79.227.41:4086 , piece_id=S6CEQVRFEGR42PXS6OZSMQ6XPA57MHJEAI6L2ZPYU67DGRGQ4YWA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.625350385 (83h7m35.49643343s) [noise=true , address=145.79.227.16:4910 , piece_id=MOZZ27FIQRWYHA5UJCOH3GKMKVICAF5WKJA5SBYYCMR6TZ2P4YUQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.626939180 (83h7m35.498022225s) [noise=true , address=145.79.227.11:4732 , piece_id=IZEWLKSKSK5GN7V7LUCRSUYBNXZR4DIJZJPHI4LXLHV7KIV7FSOQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.630536180 (83h7m35.501619225s) [noise=true , address=145.79.227.34:4907 , piece_id=HO3VDRVKFFFWL2I7TC2V5AKWFYIORAVQOGJI7I237QPDDSHVQ2WQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:35.632515828 (83h7m35.503598873s) [noise=true , address=145.79.227.39:4015 , piece_id=A7AGQIA6MJR6M4A5QL3HPLA6ITTAYXSNNDLI6XCC5KXNVRITYPUA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: <nil> Dec 9 02:36:37.566200474 (83h7m37.437283519s) [noise=true , address=77.90.31.19:4841 , piece_id=MX2HJM56PK4IWSIDPLRFYZJC6SJIEKKZJ7D2Y3PMLEQEMXTTV4MA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, address:77.90.31.19:4841): protocol: storage node overloaded, request limit: 2000 Dec 9 02:37:02.685025066 (83h8m2.556108112s) [noise=true , address=77.90.31.15:4698 , piece_id=BPN4QMAI6U2EU3RGZ45V37A2VBNKHJVZR5GJ6A77Q52QCYINJZQQ, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: expected piece hash; context deadline exceeded; EOF Dec 9 02:37:02.685361857 (83h8m2.556444892s) [noise=true , address=145.79.227.23:4546 , piece_id=YMDL4F2QYIY2OCWI3P46FSAMWYTVQXVXIFZF6WT4C44REOLSESLA, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: read tcp 188.166.225.170:44422->145.79.227.23:4546: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 9 02:37:02.686002387 (83h8m2.557085432s) [noise=true , address=145.79.227.28:4727 , piece_id=RJXEG4WHX4PZYTBWCZV6KL6O46D62GUUKAJJHSHCMIXYO2KEND6A, seg_pos={0 0}, upload=72607]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:36574->145.79.227.28:4727: use of closed network connection; context deadline exceeded Dec 9 02:37:02.686061210 (83h8m2.557144255s) [seg_pos={0 0}, upload=72607]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Dec 9 02:37:02.933792900 (83h8m2.804875945s) [upload=72607]: Done uploading segments... Dec 9 02:37:02.933926595 (83h8m2.80500964s) [upload=72607]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.771079915s core uplink us-select-1

Orbiter Check History

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)