Orbiter History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.009733826s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:07:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.009787318s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.980652911s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:06:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.980683499s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 476.093879ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:06:48 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.122914ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [X.X.X.X.....X....XXXX......XX...X........XXX...X............] duration: 3.505469071s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:06:47 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.505533124s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X............................X.........................] duration: 6.17921878s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:06:41 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.179278293s core uplink us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 692.233732ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:06:32 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 692.272808ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [.X.X.X.X.....X....XXXX......XX...X........XXX...X...........] duration: 3.490085039s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:06:29 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.490104917s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X............................X........................] duration: 5.912567793s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:06:25 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.912615412s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.407000158s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:06:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.407026651s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.915653739s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:06:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.915709835s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 10.046246425s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:05:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 10.046274129s core uplink eu-select-1 eu1-placement-15
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X............................X.......................] duration: 6.586712412s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:05:48 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.586733643s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 475.633552ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:05:43 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 475.684309ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 11.976676301s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:05:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 11.976705126s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.879529761s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:05:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.879562102s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 693.842016ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:05:25 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 693.872193ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [..X.X.X.X.....X....XXXX......XX...X........XXX...X..........] duration: 4.410234443s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:05:19 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.410267546s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 476.59882ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:05:10 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.628416ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X............................X......................] duration: 6.144858372s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:04:59 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 6.144887407s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [...X.X.X.X.....X....XXXX......XX...X........XXX...X.........] duration: 3.467967871s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:04:56 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.467995753s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.616057526s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:04:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.616086812s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.822098259s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:04:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.822130761s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 478.68326ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:04:46 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 478.713227ms core satellite eu1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 695.873382ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:04:36 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 695.904603ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.697489918s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:04:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.697520216s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 713.089536ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:04:17 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 713.137678ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.967718781s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:04:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.967743337s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [X...X.X.X.X.....X....XXXX......XX...X........XXX...X........] duration: 3.532475555s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:04:09 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.532506604s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.622792932s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:04:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.622822018s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.833506977s trace: https://jaeger.dev.storj.io/trace/20c661c94f160c60 FAIL (2025-12-08 17:03:58 UTC) orbiter warning: context deadline exceeded Duration: 5.833533668s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 704.744426ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:03:56 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 704.778961ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X............................X.....................] duration: 5.897974373s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:03:53 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.898006215s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 475.5217ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:03:51 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 475.548902ms core satellite eu1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 11.271606925s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:03:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 11.27163074s core uplink eu-select-1 eu1-placement-15
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 466.964114ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:03:29 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 467.073481ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.860995757s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:03:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.861023279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m53.764389059s trace: https://jaeger.dev.storj.io/trace/2bacc06da04f4505 FAIL (2025-12-08 17:03:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== e=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.188203846 (73h32m28.059286901s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.189671170 (73h32m28.060754215s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.191039326 (73h32m28.062122361s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.193557270 (73h32m28.064640305s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.195856628 (73h32m28.066939674s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.198414219 (73h32m28.069497264s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.201095685 (73h32m28.07217873s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:28.204972177 (73h32m28.076055212s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Uploading piece... Dec 8 17:01:29.014501224 (73h32m28.885584269s) [upload=64365]: Next returned nil segment Dec 8 17:01:29.014527022 (73h32m28.885610067s) [upload=64365]: Waiting for error group managing segments... Dec 8 17:01:29.408440684 (73h32m29.279523729s) [noise=true , address=185.167.204.205:4023 , piece_id=V344WVXRV2SEML6TEIASWYT4427PRE56R3KLFQJZ4ZNUFCQTNF7Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.409778682 (73h32m29.280861728s) [noise=true , address=185.167.204.205:4034 , piece_id=OGTA4AK7H6XFMPSB2HDYAYMVUHLMFU5IYM3TKOFWWDZBHYFAAESQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.411240316 (73h32m29.282323361s) [noise=true , address=185.167.204.206:4053 , piece_id=4MMKVKWPWU6CK5M7R74PXGQ34PPUXGWUJT2LJ6ZEXV6A2X72PJEQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.412578493 (73h32m29.283661538s) [noise=true , address=185.167.204.205:4036 , piece_id=6VYQ5BV7BKIGEIHKBERSBYRKTP7VC7M6NPNKWRSYSRVP243OVV4A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.419000773 (73h32m29.290083818s) [noise=true , address=185.167.204.205:4029 , piece_id=KNEK65KYMS5VO2REHA4UZ6WXFVKYT7LUQ47WPWDH6U6IWBYYFFXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.427501339 (73h32m29.298584384s) [noise=true , address=185.167.204.206:4068 , piece_id=4ZBFLMHY2SZWQKT6SLGWH67EYB2UYOBVFW32X3NMEKJ2QGHUB62Q, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.431109811 (73h32m29.302192856s) [noise=true , address=185.167.204.205:4030 , piece_id=I5Y6S4GGMJK3HALRUWMSKQVLZMCSD5Q34GHLOJDKEOBWY7KLRMTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432493316 (73h32m29.303576351s) [noise=true , address=185.167.204.206:4037 , piece_id=JWENQ4SQJCAD6RAEM35TQZNDXTMUOGV7JGWOLGA64BFQUF2MDKQQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.432695100 (73h32m29.303778135s) [noise=true , address=185.167.204.206:4070 , piece_id=RIILUOUAANPXHFPC67DDMBBXDRCIPMNGJKHPSGKBWPIKZLLB3SIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.433119147 (73h32m29.304202192s) [noise=true , address=185.167.204.206:4039 , piece_id=56TVAHFZJLIWEWOWZCLQML2LX4IDFAERQQXRHDI2WAQPDDS36YNA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434343859 (73h32m29.305426904s) [noise=true , address=185.167.204.206:4061 , piece_id=Y6BICZVIUK6OFLRXZJE3S4VVFIRWSAIJBAFOL6AEY66CCKZQFCTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.434887174 (73h32m29.305970209s) [noise=true , address=185.167.204.206:4064 , piece_id=NCWE447MXDE4SAN7MSMHMIU2TCDH2X7M5OJSICXKCE2Y3OFEACZA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439186210 (73h32m29.310269255s) [noise=true , address=185.167.204.206:4050 , piece_id=BETXF3XMM5F42JB44YF2QGK6N54JOA6CZ4Z2EV5WOXZPUD2KKKTA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.439715649 (73h32m29.310798684s) [noise=true , address=185.167.204.206:4069 , piece_id=XGOYKYXSUAK4IHEM44RZB4DHNXYXIYQTVWZDNAPNE5G5HWUK4VFA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440245988 (73h32m29.311329033s) [noise=true , address=185.167.204.205:4001 , piece_id=XAESPYQ55LBDJQ7WJIFWCVTCAGNXWAOOMSL3VMRERS3LZTMZB3GQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.440484813 (73h32m29.311567848s) [noise=true , address=185.167.204.206:4049 , piece_id=2NHQKCEPOXX475AS5SIZ7IXCNKJMXWW6LNU2OO6PBILRZ6DUZ5EA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.442183288 (73h32m29.313266323s) [noise=true , address=185.167.204.205:4015 , piece_id=FJKFMAGYH277I25J3WQ3IWR3DYRWICWQRC5LSKDDJ63XZ74S7MXQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.444441307 (73h32m29.315524352s) [noise=true , address=185.167.204.206:4045 , piece_id=DDDPLG5LX3S7NJZDZSB4NQTMY4KBHBN3N5UGREBDB5JJHKBW3WWQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.445075635 (73h32m29.31615868s) [noise=true , address=185.167.204.205:4025 , piece_id=6VYO47IF26LZ5LO7KSUMRFMWOL6PKBOGT75UQQGCQUYX6CEFOVIQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.448313100 (73h32m29.319396135s) [noise=true , address=185.167.204.206:4048 , piece_id=BU4IJLUUZ66CLPARWCCMRZCCDQYHYWZOGK2EASU3HRNWLQ7B5RUA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.450008007 (73h32m29.321091042s) [noise=true , address=185.167.204.206:4057 , piece_id=ZLIWTLFQD4KDPMWOKRZUAVTK4BUF2J3BH6ETCMWBC7HMN5JBUQJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:29.464765288 (73h32m29.335848323s) [noise=true , address=185.167.204.205:4007 , piece_id=45SASPPJTGKEBQTAINBPGKBUCTPJH4QXLGP34TK5H6MF5BK5AUAA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.006798349 (73h32m29.877881394s) [noise=true , address=81.171.14.70:4044 , piece_id=I4DPAS3D7CK5K5RMS7KASK775IF2QTC5SLDRNEXERI7CPHACCSMA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.052457282 (73h32m29.923540327s) [noise=true , address=81.171.14.70:4026 , piece_id=KL7AYBMWQ5P63JLIQPHVHBVZOFTOMSS2XJF2QTTIRQGF7ROTYPTQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.087401313 (73h32m29.958484358s) [noise=true , address=81.171.14.70:4027 , piece_id=3XX2VTW3KPECPWINTSTFFUQL5CRKO4GENBS2TVTNSIN44YXCETJA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150558347 (73h32m30.021641392s) [noise=true , address=81.171.14.70:4032 , piece_id=ZMDAJH2OX64GP4YR5E7CBNJC72RDTRMTCXEW67KIGQIZHJBO3ECQ, seg_pos={0 0}, upload=64365]: Done uploading piece. err: <nil> Dec 8 17:01:30.150583825 (73h32m30.02166687s) [seg_pos={0 0}, upload=64365]: Segment reached optimal threshold of 26 pieces. Dec 8 17:01:30.150889357 (73h32m30.021972412s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XsvrVhDGkU8BkYsBGmTtax47E43dZAYuGwD8WBMpp3n4GgDM5): protocol: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33966->89.149.201.147:4016: use of closed network connection Dec 8 17:01:30.150901891 (73h32m30.021984936s) [noise=true , address=89.149.201.147:4016 , piece_id=BQT6TNPFUVT2OTT4YIRIC3CNAPIANWHIT23W5JPPLK6Y542ZER7A, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150914835 (73h32m30.02199788s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12PcxodnQMJERGjvBgnz3iEiC9MseRhMydkCriwGpQKqi6DGFDg): protocol: read tcp 188.166.225.170:55962->89.149.201.147:4011: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Dec 8 17:01:30.150922571 (73h32m30.022005626s) [noise=true , address=89.149.201.147:4011 , piece_id=BEU4ZFPKV5CFS3EU5FL5IOBSR2462IN5SUPKAEAWPGAHXWELR5AA, seg_pos={0 0}, upload=64365]: Long tail cancellation due to optimal threshold Dec 8 17:01:30.150944682 (73h32m30.022027727s) [seg_pos={0 0}, upload=64365]: Done waiting for segment. successful: 26 optimal: 26 errs: <nil> Dec 8 17:01:30.151058490 (73h32m30.022141535s) [upload=64365]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 17:01:30.151066825 (73h32m30.02214987s) [upload=64365]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 17:03:21.289875989 (73h34m21.160959045s) [upload=64365]: Done uploading segments... Dec 8 17:03:21.289984466 (73h34m21.161067531s) [upload=64365]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m53.764409388s core uplink eu-select-1 eu1-placement-15
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [XX...X.X.X.X.....X....XXXX......XX...X........XXX...X.......] duration: 3.478736018s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:03:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.478785041s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.743510062s trace: https://jaeger.dev.storj.io/trace/20c661c94f160c60 FAIL (2025-12-08 17:03:03 UTC) orbiter warning: context deadline exceeded Duration: 5.743538296s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 700.128196ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:03:01 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 700.185875ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.750891709s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:03:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.750925013s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 482.236316ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:02:58 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 482.284918ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X............................X....................] duration: 5.836633872s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:02:44 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.836667326s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 5.917552779s trace: https://jaeger.dev.storj.io/trace/20c661c94f160c60 FAIL (2025-12-08 17:02:44 UTC) orbiter warning: context deadline exceeded Duration: 5.917578978s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 715.878789ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/76a069d29660539a FAIL (2025-12-08 17:02:42 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 715.947679ms core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 476.375805ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/6c02152d3764cef5 FAIL (2025-12-08 17:02:37 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.431951ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X............................X...................] duration: 5.915890399s trace: https://jaeger.dev.storj.io/trace/418daf3273797203 FAIL (2025-12-08 17:02:15 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 5.915915287s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 18 out of 60 is failed (threshold=6) [.XX...X.X.X.X.....X....XXXX......XX...X........XXX...X......] duration: 3.523123368s trace: https://jaeger.dev.storj.io/trace/3de63f0ed71cc7b9 FAIL (2025-12-08 17:02:15 UTC) orbiter warning: operation error S3: GetObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.523153815s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.631991864s trace: https://jaeger.dev.storj.io/trace/705a9a7864a3a1cc FAIL (2025-12-08 17:02:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.632022373s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 5.844947205s trace: https://jaeger.dev.storj.io/trace/20c661c94f160c60 FAIL (2025-12-08 17:02:06 UTC) orbiter warning: context deadline exceeded Duration: 5.84498191s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks-placement-15", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.placement_15)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)