Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 870286 reports (Page 11593 of 17406)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 715.615947ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:14:29 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 715.642759ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [XX..................X....................XX............X...X] duration: 30.001579665s trace: https://jaeger.dev.storj.io/trace/15d42e97277aab82 FAIL (2025-12-09 16:14:29 UTC) orbiter warning: context deadline exceeded Duration: 30.001606576s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 1.02336234s remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:14:07 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 1.023390404s core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.143578131s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:14:05 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.14364163s core uplink us-select-1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 476.650992ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/39a3ed12475253d0 FAIL (2025-12-09 16:14:05 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 476.677862ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X...........X..................................] duration: 5.771909072s trace: https://jaeger.dev.storj.io/trace/593b271687016c4f FAIL (2025-12-09 16:13:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 15:47:50.798824866 (96h18m50.669907901s) [upload=83937]: Starting upload Dec 9 15:47:50.798921369 (96h18m50.670004404s) [upload=83937]: Done starting upload Dec 9 15:47:50.799010178 (96h18m50.670093213s) [upload=83937]: Uploading segments... Dec 9 15:47:50.799294159 (96h18m50.670377204s) [upload=83937]: Got next segment. Inline: false Dec 9 15:47:50.799363250 (96h18m50.670446295s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 15:47:50.799372037 (96h18m50.670455092s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 15:47:51.056426954 (96h18m50.92751001s) [seg_pos={0 0}, upload=83937]: Begin upload segment... Dec 9 15:47:51.056481187 (96h18m50.927564232s) [seg_pos={0 0}, upload=83937]: Done begin upload segment. Dec 9 15:47:51.056513298 (96h18m50.927596343s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.056576559 (96h18m50.927659594s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057100897 (96h18m50.928183942s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057596040 (96h18m50.928679075s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058125198 (96h18m50.929208243s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058421582 (96h18m50.929504617s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058696356 (96h18m50.929779391s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058972071 (96h18m50.930055106s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.059347686 (96h18m50.930430721s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.061182009 (96h18m50.932265044s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.062514146 (96h18m50.933597181s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.064197953 (96h18m50.935280998s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.065677489 (96h18m50.936760534s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.067090440 (96h18m50.938173485s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.068340300 (96h18m50.939423345s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.069558109 (96h18m50.940641154s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:52.703886775 (96h18m52.57496981s) [upload=83937]: Next returned nil segment Dec 9 15:47:52.703900601 (96h18m52.574983636s) [upload=83937]: Waiting for error group managing segments... Dec 9 15:47:53.070939400 (96h18m52.942022445s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.074549104 (96h18m52.945632139s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077053072 (96h18m52.948136107s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077574054 (96h18m52.948657099s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.084535460 (96h18m52.955618505s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.090700220 (96h18m52.961783265s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.091191014 (96h18m52.962274079s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094302989 (96h18m52.965386025s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094537586 (96h18m52.965620622s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.115109921 (96h18m52.986192966s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.118417168 (96h18m52.989500213s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.120155649 (96h18m52.991238684s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123155120 (96h18m52.994238155s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123168146 (96h18m52.994251181s) [seg_pos={0 0}, upload=83937]: Segment reached optimal threshold of 13 pieces. Dec 9 15:47:53.123387333 (96h18m52.994470368s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123397562 (96h18m52.994480597s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123682554 (96h18m52.994765599s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123691131 (96h18m52.994774176s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123939985 (96h18m52.99502303s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123949333 (96h18m52.995032378s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123987134 (96h18m52.99507017s) [seg_pos={0 0}, upload=83937]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 9 15:47:53.124075293 (96h18m52.995158338s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 15:47:53.124081745 (96h18m52.99516479s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 15:48:17.417713882 (96h19m17.288796927s) [upload=83937]: Done uploading segments... Dec 9 15:48:17.417734421 (96h19m17.288817456s) [upload=83937]: Upload finished. err: <nil> Duration: 5.77194003s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.........................X..........................X.......] duration: 10.721574135s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:13:54 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 10.721604093s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.XX..................X....................XX............X...] duration: 26.106186188s trace: https://jaeger.dev.storj.io/trace/474316135cb6f376 FAIL (2025-12-09 16:13:47 UTC) orbiter warning: context deadline exceeded Duration: 26.106216857s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [.X...X...X.X..XX...XX.X......X.X.X.....X...X....X...X....X..] duration: 4.578463488s trace: https://jaeger.dev.storj.io/trace/77817e8a9a38e7ff FAIL (2025-12-09 16:13:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.578492192s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 711.136028ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:13:40 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 711.165475ms core satellite us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..........................X..........................X......] duration: 5.770535566s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:13:24 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.770566807s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 711.447611ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:13:14 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 711.476506ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [..X...X...X.X..XX...XX.X......X.X.X.....X...X....X...X....X.] duration: 3.596431799s trace: https://jaeger.dev.storj.io/trace/77817e8a9a38e7ff FAIL (2025-12-09 16:13:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.596458359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.088204437s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:13:11 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.088245064s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...........................X..........................X.....] duration: 5.680346978s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:13:03 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.680382585s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X...........X.................................] duration: 5.916900975s trace: https://jaeger.dev.storj.io/trace/593b271687016c4f FAIL (2025-12-09 16:13:02 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 15:47:50.798824866 (96h18m50.669907901s) [upload=83937]: Starting upload Dec 9 15:47:50.798921369 (96h18m50.670004404s) [upload=83937]: Done starting upload Dec 9 15:47:50.799010178 (96h18m50.670093213s) [upload=83937]: Uploading segments... Dec 9 15:47:50.799294159 (96h18m50.670377204s) [upload=83937]: Got next segment. Inline: false Dec 9 15:47:50.799363250 (96h18m50.670446295s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 15:47:50.799372037 (96h18m50.670455092s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 15:47:51.056426954 (96h18m50.92751001s) [seg_pos={0 0}, upload=83937]: Begin upload segment... Dec 9 15:47:51.056481187 (96h18m50.927564232s) [seg_pos={0 0}, upload=83937]: Done begin upload segment. Dec 9 15:47:51.056513298 (96h18m50.927596343s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.056576559 (96h18m50.927659594s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057100897 (96h18m50.928183942s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057596040 (96h18m50.928679075s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058125198 (96h18m50.929208243s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058421582 (96h18m50.929504617s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058696356 (96h18m50.929779391s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058972071 (96h18m50.930055106s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.059347686 (96h18m50.930430721s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.061182009 (96h18m50.932265044s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.062514146 (96h18m50.933597181s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.064197953 (96h18m50.935280998s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.065677489 (96h18m50.936760534s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.067090440 (96h18m50.938173485s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.068340300 (96h18m50.939423345s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.069558109 (96h18m50.940641154s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:52.703886775 (96h18m52.57496981s) [upload=83937]: Next returned nil segment Dec 9 15:47:52.703900601 (96h18m52.574983636s) [upload=83937]: Waiting for error group managing segments... Dec 9 15:47:53.070939400 (96h18m52.942022445s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.074549104 (96h18m52.945632139s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077053072 (96h18m52.948136107s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077574054 (96h18m52.948657099s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.084535460 (96h18m52.955618505s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.090700220 (96h18m52.961783265s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.091191014 (96h18m52.962274079s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094302989 (96h18m52.965386025s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094537586 (96h18m52.965620622s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.115109921 (96h18m52.986192966s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.118417168 (96h18m52.989500213s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.120155649 (96h18m52.991238684s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123155120 (96h18m52.994238155s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123168146 (96h18m52.994251181s) [seg_pos={0 0}, upload=83937]: Segment reached optimal threshold of 13 pieces. Dec 9 15:47:53.123387333 (96h18m52.994470368s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123397562 (96h18m52.994480597s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123682554 (96h18m52.994765599s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123691131 (96h18m52.994774176s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123939985 (96h18m52.99502303s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123949333 (96h18m52.995032378s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123987134 (96h18m52.99507017s) [seg_pos={0 0}, upload=83937]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 9 15:47:53.124075293 (96h18m52.995158338s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 15:47:53.124081745 (96h18m52.99516479s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 15:48:17.417713882 (96h19m17.288796927s) [upload=83937]: Done uploading segments... Dec 9 15:48:17.417734421 (96h19m17.288817456s) [upload=83937]: Upload finished. err: <nil> Duration: 5.916933696s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 713.261876ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:12:57 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 713.288106ms core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 469.175156ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/39a3ed12475253d0 FAIL (2025-12-09 16:12:52 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 469.204651ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.035293532s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:12:51 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.035325101s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..XX..................X....................XX............X..] duration: 29.825334322s trace: https://jaeger.dev.storj.io/trace/474316135cb6f376 FAIL (2025-12-09 16:12:50 UTC) orbiter warning: context deadline exceeded Duration: 29.825362334s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [............................X..........................X....] duration: 9.541269866s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:12:46 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 9.541299943s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 706.470895ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:12:33 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 706.49468ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [...X...X...X.X..XX...XX.X......X.X.X.....X...X....X...X....X] duration: 5.001132184s trace: https://jaeger.dev.storj.io/trace/77817e8a9a38e7ff FAIL (2025-12-09 16:12:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.001156671s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...XX..................X....................XX............X.] duration: 20.267946714s trace: https://jaeger.dev.storj.io/trace/474316135cb6f376 FAIL (2025-12-09 16:12:18 UTC) orbiter warning: context deadline exceeded Duration: 20.267979618s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [.............................X..........................X...] duration: 17.724238187s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:12:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 17.724268615s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 709.171177ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:11:52 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 709.199551ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X...........X................................] duration: 5.7794296s trace: https://jaeger.dev.storj.io/trace/593b271687016c4f FAIL (2025-12-09 16:11:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 15:47:50.798824866 (96h18m50.669907901s) [upload=83937]: Starting upload Dec 9 15:47:50.798921369 (96h18m50.670004404s) [upload=83937]: Done starting upload Dec 9 15:47:50.799010178 (96h18m50.670093213s) [upload=83937]: Uploading segments... Dec 9 15:47:50.799294159 (96h18m50.670377204s) [upload=83937]: Got next segment. Inline: false Dec 9 15:47:50.799363250 (96h18m50.670446295s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 15:47:50.799372037 (96h18m50.670455092s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 15:47:51.056426954 (96h18m50.92751001s) [seg_pos={0 0}, upload=83937]: Begin upload segment... Dec 9 15:47:51.056481187 (96h18m50.927564232s) [seg_pos={0 0}, upload=83937]: Done begin upload segment. Dec 9 15:47:51.056513298 (96h18m50.927596343s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.056576559 (96h18m50.927659594s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057100897 (96h18m50.928183942s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057596040 (96h18m50.928679075s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058125198 (96h18m50.929208243s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058421582 (96h18m50.929504617s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058696356 (96h18m50.929779391s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058972071 (96h18m50.930055106s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.059347686 (96h18m50.930430721s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.061182009 (96h18m50.932265044s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.062514146 (96h18m50.933597181s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.064197953 (96h18m50.935280998s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.065677489 (96h18m50.936760534s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.067090440 (96h18m50.938173485s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.068340300 (96h18m50.939423345s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.069558109 (96h18m50.940641154s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:52.703886775 (96h18m52.57496981s) [upload=83937]: Next returned nil segment Dec 9 15:47:52.703900601 (96h18m52.574983636s) [upload=83937]: Waiting for error group managing segments... Dec 9 15:47:53.070939400 (96h18m52.942022445s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.074549104 (96h18m52.945632139s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077053072 (96h18m52.948136107s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077574054 (96h18m52.948657099s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.084535460 (96h18m52.955618505s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.090700220 (96h18m52.961783265s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.091191014 (96h18m52.962274079s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094302989 (96h18m52.965386025s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094537586 (96h18m52.965620622s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.115109921 (96h18m52.986192966s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.118417168 (96h18m52.989500213s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.120155649 (96h18m52.991238684s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123155120 (96h18m52.994238155s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123168146 (96h18m52.994251181s) [seg_pos={0 0}, upload=83937]: Segment reached optimal threshold of 13 pieces. Dec 9 15:47:53.123387333 (96h18m52.994470368s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123397562 (96h18m52.994480597s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123682554 (96h18m52.994765599s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123691131 (96h18m52.994774176s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123939985 (96h18m52.99502303s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123949333 (96h18m52.995032378s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123987134 (96h18m52.99507017s) [seg_pos={0 0}, upload=83937]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 9 15:47:53.124075293 (96h18m52.995158338s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 15:47:53.124081745 (96h18m52.99516479s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 15:48:17.417713882 (96h19m17.288796927s) [upload=83937]: Done uploading segments... Dec 9 15:48:17.417734421 (96h19m17.288817456s) [upload=83937]: Upload finished. err: <nil> Duration: 5.7794494s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 16 out of 60 is failed (threshold=6) [....X...X...X.X..XX...XX.X......X.X.X.....X...X....X...X....] duration: 3.553578116s trace: https://jaeger.dev.storj.io/trace/4e2f19670640c087 FAIL (2025-12-09 16:11:49 UTC) orbiter warning: context deadline exceeded Duration: 3.553610117s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 470.387835ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/39a3ed12475253d0 FAIL (2025-12-09 16:11:41 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 470.407212ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.062001589s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:11:37 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.062035575s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....XX..................X....................XX............X] duration: 30.000830874s trace: https://jaeger.dev.storj.io/trace/474316135cb6f376 FAIL (2025-12-09 16:11:34 UTC) orbiter warning: context deadline exceeded Duration: 30.000855039s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [..............................X..........................X..] duration: 11.569048555s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:11:28 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 11.569075476s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 717.50235ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:11:05 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 717.537308ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................X...........X...............................] duration: 5.737869023s trace: https://jaeger.dev.storj.io/trace/593b271687016c4f FAIL (2025-12-09 16:10:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 15:47:50.798824866 (96h18m50.669907901s) [upload=83937]: Starting upload Dec 9 15:47:50.798921369 (96h18m50.670004404s) [upload=83937]: Done starting upload Dec 9 15:47:50.799010178 (96h18m50.670093213s) [upload=83937]: Uploading segments... Dec 9 15:47:50.799294159 (96h18m50.670377204s) [upload=83937]: Got next segment. Inline: false Dec 9 15:47:50.799363250 (96h18m50.670446295s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 15:47:50.799372037 (96h18m50.670455092s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 15:47:51.056426954 (96h18m50.92751001s) [seg_pos={0 0}, upload=83937]: Begin upload segment... Dec 9 15:47:51.056481187 (96h18m50.927564232s) [seg_pos={0 0}, upload=83937]: Done begin upload segment. Dec 9 15:47:51.056513298 (96h18m50.927596343s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.056576559 (96h18m50.927659594s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057100897 (96h18m50.928183942s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057596040 (96h18m50.928679075s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058125198 (96h18m50.929208243s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058421582 (96h18m50.929504617s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058696356 (96h18m50.929779391s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058972071 (96h18m50.930055106s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.059347686 (96h18m50.930430721s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.061182009 (96h18m50.932265044s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.062514146 (96h18m50.933597181s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.064197953 (96h18m50.935280998s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.065677489 (96h18m50.936760534s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.067090440 (96h18m50.938173485s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.068340300 (96h18m50.939423345s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.069558109 (96h18m50.940641154s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:52.703886775 (96h18m52.57496981s) [upload=83937]: Next returned nil segment Dec 9 15:47:52.703900601 (96h18m52.574983636s) [upload=83937]: Waiting for error group managing segments... Dec 9 15:47:53.070939400 (96h18m52.942022445s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.074549104 (96h18m52.945632139s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077053072 (96h18m52.948136107s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077574054 (96h18m52.948657099s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.084535460 (96h18m52.955618505s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.090700220 (96h18m52.961783265s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.091191014 (96h18m52.962274079s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094302989 (96h18m52.965386025s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094537586 (96h18m52.965620622s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.115109921 (96h18m52.986192966s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.118417168 (96h18m52.989500213s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.120155649 (96h18m52.991238684s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123155120 (96h18m52.994238155s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123168146 (96h18m52.994251181s) [seg_pos={0 0}, upload=83937]: Segment reached optimal threshold of 13 pieces. Dec 9 15:47:53.123387333 (96h18m52.994470368s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123397562 (96h18m52.994480597s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123682554 (96h18m52.994765599s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123691131 (96h18m52.994774176s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123939985 (96h18m52.99502303s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123949333 (96h18m52.995032378s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123987134 (96h18m52.99507017s) [seg_pos={0 0}, upload=83937]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 9 15:47:53.124075293 (96h18m52.995158338s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 15:47:53.124081745 (96h18m52.99516479s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 15:48:17.417713882 (96h19m17.288796927s) [upload=83937]: Done uploading segments... Dec 9 15:48:17.417734421 (96h19m17.288817456s) [upload=83937]: Upload finished. err: <nil> Duration: 5.73789908s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.623358836s trace: https://jaeger.dev.storj.io/trace/2764ea1e2faac6a FAIL (2025-12-09 16:10:47 UTC) orbiter warning: context deadline exceeded Duration: 6.623460009s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 17 out of 60 is failed (threshold=6) [X....X...X...X.X..XX...XX.X......X.X.X.....X...X....X...X...] duration: 3.500690129s trace: https://jaeger.dev.storj.io/trace/4e2f19670640c087 FAIL (2025-12-09 16:10:42 UTC) orbiter warning: context deadline exceeded Duration: 3.500708043s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [...............................X..........................X.] duration: 5.19860147s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:10:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.198627689s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 39.874469511s trace: https://jaeger.dev.storj.io/trace/2764ea1e2faac6a FAIL (2025-12-09 16:10:40 UTC) orbiter warning: context deadline exceeded Duration: 39.874498015s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 475.120086ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/39a3ed12475253d0 FAIL (2025-12-09 16:10:36 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 475.149912ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.131260914s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:10:27 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.131316411s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 10.133749611s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:10:19 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 10.133832299s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.....XX..................X....................XX............] duration: 20.483943108s trace: https://jaeger.dev.storj.io/trace/58029074a3b7a1f6 FAIL (2025-12-09 16:10:11 UTC) orbiter warning: context deadline exceeded Duration: 20.483977002s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (gateway.us1.inline) antiblip: 2 out of 60 is failed (threshold=3) [................................X..........................X] duration: 12.677887461s trace: https://jaeger.dev.storj.io/trace/6cc223a408573522 FAIL (2025-12-09 16:10:09 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 12.677913821s edge gateway us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 714.312256ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:10:08 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 714.339398ms core satellite us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 475.627351ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/39a3ed12475253d0 FAIL (2025-12-09 16:09:55 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 475.656638ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................X...........X..............................] duration: 5.816068885s trace: https://jaeger.dev.storj.io/trace/593b271687016c4f FAIL (2025-12-09 16:09:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Dec 9 15:47:50.798824866 (96h18m50.669907901s) [upload=83937]: Starting upload Dec 9 15:47:50.798921369 (96h18m50.670004404s) [upload=83937]: Done starting upload Dec 9 15:47:50.799010178 (96h18m50.670093213s) [upload=83937]: Uploading segments... Dec 9 15:47:50.799294159 (96h18m50.670377204s) [upload=83937]: Got next segment. Inline: false Dec 9 15:47:50.799363250 (96h18m50.670446295s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectBegin Dec 9 15:47:50.799372037 (96h18m50.670455092s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentBegin Dec 9 15:47:51.056426954 (96h18m50.92751001s) [seg_pos={0 0}, upload=83937]: Begin upload segment... Dec 9 15:47:51.056481187 (96h18m50.927564232s) [seg_pos={0 0}, upload=83937]: Done begin upload segment. Dec 9 15:47:51.056513298 (96h18m50.927596343s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.056576559 (96h18m50.927659594s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057100897 (96h18m50.928183942s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.057596040 (96h18m50.928679075s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058125198 (96h18m50.929208243s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058421582 (96h18m50.929504617s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058696356 (96h18m50.929779391s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.058972071 (96h18m50.930055106s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.059347686 (96h18m50.930430721s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.061182009 (96h18m50.932265044s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.062514146 (96h18m50.933597181s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.064197953 (96h18m50.935280998s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.065677489 (96h18m50.936760534s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.067090440 (96h18m50.938173485s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.068340300 (96h18m50.939423345s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:51.069558109 (96h18m50.940641154s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Uploading piece... Dec 9 15:47:52.703886775 (96h18m52.57496981s) [upload=83937]: Next returned nil segment Dec 9 15:47:52.703900601 (96h18m52.574983636s) [upload=83937]: Waiting for error group managing segments... Dec 9 15:47:53.070939400 (96h18m52.942022445s) [noise=true , address=145.79.227.44:4194 , piece_id=6NCPCDJVVV7F2HY4J6MGR3TD6MOHETXWAUQ7H3FU43AGIBZMBWUA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.074549104 (96h18m52.945632139s) [noise=true , address=145.79.227.39:4014 , piece_id=TNJNQ7DJCHPKOMP2KAW75ZCOMHOBPPWH4XX3WQP5OGZRMCIYHVEA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077053072 (96h18m52.948136107s) [noise=true , address=145.79.227.39:4013 , piece_id=IC654IOYAZRRRKM7G34AXHTQA2QNFBWUP53G3Q4KAXCZOKPI6ZNA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.077574054 (96h18m52.948657099s) [noise=true , address=145.79.227.38:4979 , piece_id=BLERVPIAPSSU7VR26NMIQBRHNUCWLPNYI6E5YD52IBM2JV4CSFEQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.084535460 (96h18m52.955618505s) [noise=true , address=145.79.227.16:4911 , piece_id=7FCNYCTIFXMYEQWKFDU5QVN5XFKSO32RVQTVUDPRIZQCS52I4SBQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.090700220 (96h18m52.961783265s) [noise=true , address=145.79.227.27:4691 , piece_id=7TOCUCNX4ZFERAV2USO5TKQ37JL7E6B3WCYJK4DNPWH4GBC2JHJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.091191014 (96h18m52.962274079s) [noise=true , address=145.79.227.7:4294 , piece_id=NK4AXP6SSK37JGO7OINKL3BH4D23Q5VCGBACQENBPA3HRZ3EMJJA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094302989 (96h18m52.965386025s) [noise=true , address=145.79.227.32:4835 , piece_id=TQB4GDUDDWEYTAUHTDLNVNECRPU5UJWMALOL7KU2S6T4GBR5PM3Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.094537586 (96h18m52.965620622s) [noise=true , address=145.79.227.29:4941 , piece_id=WNUYTTG2LWEKL64R35VQUM7WF4WKLOED4NFFKLY4LPEU27WBMAGA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.115109921 (96h18m52.986192966s) [noise=true , address=145.79.227.42:4122 , piece_id=W5A5G7N2J4775C2TUGUFHC2SR7OSA3AQRMDHSLKTS5K2XC3EHCZQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.118417168 (96h18m52.989500213s) [noise=true , address=145.79.227.43:4158 , piece_id=5FKHKDI3C64XWOUFMNNUSG5PPGK4A4HNX3MWMP4E44CW2BYTBP4A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.120155649 (96h18m52.991238684s) [noise=true , address=145.79.227.23:4547 , piece_id=7LLDJKIMPTDCDLTDARQXO3CZYVCD3OZDD7NOHD7EGBP6XUSPXKGQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123155120 (96h18m52.994238155s) [noise=true , address=77.90.31.14:4661 , piece_id=DGHMAMD7ASLQLLW26A2A4L2KVFD5N2FC5GNJO7UP4W3PFBUHU56Q, seg_pos={0 0}, upload=83937]: Done uploading piece. err: <nil> Dec 9 15:47:53.123168146 (96h18m52.994251181s) [seg_pos={0 0}, upload=83937]: Segment reached optimal threshold of 13 pieces. Dec 9 15:47:53.123387333 (96h18m52.994470368s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123397562 (96h18m52.994480597s) [noise=true , address=77.90.31.21:4879 , piece_id=XDXZ7SACIVSTVQWL5JE4J2XI23UANBGE3RFW7EFL7GLHPPTVPUJQ, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123682554 (96h18m52.994765599s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1n91vwhc11FPE57QAUCfgDpffrmPrJxWxh7R9ecgt41JEPux6J): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123691131 (96h18m52.994774176s) [noise=true , address=145.79.227.40:4049 , piece_id=MMRUH72XOW5RIXQTVY75LTIB7OHUD7WUUGXRNTNNWYQQ4AJMWWBA, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123939985 (96h18m52.99502303s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Dec 9 15:47:53.123949333 (96h18m52.995032378s) [noise=true , address=77.90.31.21:4878 , piece_id=FEVDKBVRLUKDLP77PHAXUUZWE24ARARZ5EOXJCTWZOGY7ME3GA3A, seg_pos={0 0}, upload=83937]: Long tail cancellation due to optimal threshold Dec 9 15:47:53.123987134 (96h18m52.99507017s) [seg_pos={0 0}, upload=83937]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Dec 9 15:47:53.124075293 (96h18m52.995158338s) [upload=83937]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 9 15:47:53.124081745 (96h18m52.99516479s) [upload=83937]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 9 15:48:17.417713882 (96h19m17.288796927s) [upload=83937]: Done uploading segments... Dec 9 15:48:17.417734421 (96h19m17.288817456s) [upload=83937]: Upload finished. err: <nil> Duration: 5.816098653s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 718.155824ms remote: "tcp:35.212.10.183:443" target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/18d754292824c46e FAIL (2025-12-09 16:09:42 UTC) orbiter warning: Get "https://us1.storj.io/login": EOF Duration: 718.183667ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us_select_1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 2.093926864s trace: https://jaeger.dev.storj.io/trace/216abfadf4eb4969 FAIL (2025-12-09 16:09:37 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 2.093985035s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [......XX..................X....................XX...........] duration: 24.981507588s trace: https://jaeger.dev.storj.io/trace/58029074a3b7a1f6 FAIL (2025-12-09 16:09:37 UTC) orbiter warning: context deadline exceeded Duration: 24.981539748s edge gateway us-select-1
checkers.HTTP{"GET" "https://www.storj.io/dcs-satellites", ExpectedStatus: 301, ExpectedRedirect: "https://static.storj.io/dcs-satellites"} failure (antiblip window 6/15): (http.www_dcs_satellites_redirect) antiblip: 1 out of 15 is failed (threshold=6) [X..............] duration: 323.676507ms remote: "tcp:[2620:cb:2000::1]:443" response headers: Referrer-Policy=same-origin Cache-Control=private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0 Vary=Accept-Encoding Set-Cookie=_cfuvid=TLwZ_FmVd9MFS2_TET5etxP.gKrQsSv2cO5Mp07gx2A-1765295892528-0.0.1.1-604800000; path=/; domain=.www.storj.io; HttpOnly; Secure; SameSite=None Cf-Ray=9ab5b3dfbda65fc8-SIN Date=Tue, 09 Dec 2025 15:58:12 GMT X-Frame-Options=SAMEORIGIN Expires=Thu, 01 Jan 1970 00:00:01 GMT Server=cloudflare Alt-Svc=h3=":443"; ma=86400 Content-Type=text/html; charset=UTF-8 Content-Length=4316 target: https://www.storj.io/dcs-satellites trace: https://jaeger.dev.storj.io/trace/11761dcce8efd075 FAIL (2025-12-09 16:09:34 UTC) orbiter warning: unexpected status code "500 Internal Server Error" "<!DOCTYPE html>\n<!--[if lt IE 7]> <html class=\"no-js ie6 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 7]> <html class=\"no-js ie7 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if IE 8]> <html class=\"no-js ie8 oldie\" lang=\"en-US\"> <![endif]-->\n<!--[if gt IE 8]><!--> <html class=\"no-js\" lang=\"en-US\"> <!--<![endif]-->\n<head>\n<title>Worker threw exception \| www.storj.io \| Cloudflare</title>\n<meta charset=\"UTF-8\" />\n<meta http-equiv=\"Content-Type\" content=\"text/html; charset=UTF-8\" />\n<meta http-equiv=\"X-UA-Compatible\" content=\"IE=Edge\" />\n<meta name=\"robots\" content=\"noindex, nofollow\" />\n<meta name=\"viewport\" content=\"width=device-width,initial-scale=1\" />\n<link rel=\"stylesheet\" id=\"cf_styles-css\" href=\"/cdn-cgi/styles/cf.errors.css\" />\n<!--[if lt IE 9]><link rel=\"stylesheet\" id='cf_styles-ie-css' href=\"/cdn-cgi/styles/cf.errors.ie.css\" /><![endif]-->\n<style>body{margin:0;padding:0}</style>\n\n\n<!--[if gte IE 10]><!-->\n<script>\n if (!navigator.cookieEnabled) {\n window.addEventListener('DO" Duration: 323.699722ms core website

1
...
11591
11592
11593
11594
11595
...
17406