Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 523615 reports (Page 5234 of 10473)
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................]
duration:
686.735825ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:48:23 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 686.758748ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................]
duration:
692.921485ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:47:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 692.954717ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................]
duration:
685.636138ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:47:25 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 685.745997ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration:
695.661582ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:46:41 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 695.696498ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................]
duration:
684.023529ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:46:15 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 684.049488ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................]
duration:
684.251866ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:45:52 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 684.305336ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration:
710.4598ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:44:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 710.49136ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration:
701.748352ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:44:19 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 701.765224ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................]
duration:
697.277278ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:44:02 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 697.310261ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration:
692.494872ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:42:55 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 692.52492ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration:
693.229799ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:42:27 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 693.304731ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................]
duration:
707.751363ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:41:43 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 707.792892ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................]
duration:
699.511594ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:40:46 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 699.539817ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................]
duration:
698.299162ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:40:27 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 698.330022ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................]
duration:
695.24569ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:39:17 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 695.277811ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................]
duration:
685.404441ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:38:13 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 685.432023ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................]
duration:
700.044857ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:37:24 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 700.065317ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................]
duration:
693.548528ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:36:44 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 693.582513ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.............................X..............................]
duration:
692.606453ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:35:33 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 692.641079ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..............................X.............................]
duration:
702.81388ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:34:30 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 702.860719ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................]
duration:
687.702389ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:33:34 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 687.732466ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
693.578099ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:32:48 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 693.604989ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
695.889219ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:31:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 695.917443ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
685.232586ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:30:55 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 685.265758ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
687.552414ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:30:07 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 687.571721ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
703.023583ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:29:40 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 703.055895ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
691.632063ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:29:15 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 691.710101ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
692.087138ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:28:02 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 692.119098ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
695.995571ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:27:16 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 696.029256ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
692.937467ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:26:30 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 692.961624ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
695.364458ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:25:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 695.3862ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
686.554394ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:25:05 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 686.588849ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
690.487042ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:24:24 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 690.517742ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
698.570194ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:23:58 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 698.604679ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
708.612125ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:22:54 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 708.638185ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
690.216627ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:22:34 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 690.316557ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
702.01315ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:21:24 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 702.043356ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
688.147787ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:20:50 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 688.172414ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
689.964265ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:19:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 689.996336ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
701.929761ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:18:41 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 701.961089ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
684.476146ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:18:11 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 684.53129ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
693.958211ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:17:20 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 693.992247ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
695.247139ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:16:39 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 695.332281ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................]
duration:
5.989138318s
trace:
https://jaeger.dev.storj.io/trace/55ff491f49891b49
FAIL (2025-11-25 17:16:03 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 25 16:28:47.321164002 (392h24m45.872634453s) [upload=345561]: Starting upload
Nov 25 16:28:47.321244084 (392h24m45.872714535s) [upload=345561]: Done starting upload
Nov 25 16:28:47.321342281 (392h24m45.872812742s) [upload=345561]: Uploading segments...
Nov 25 16:28:47.321480945 (392h24m45.872951396s) [upload=345561]: Got next segment. Inline: false
Nov 25 16:28:47.321521653 (392h24m45.872992114s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 25 16:28:47.321528445 (392h24m45.872998897s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 25 16:28:47.569226625 (392h24m46.120697086s) [seg_pos={0 0}, upload=345561]: Begin upload segment...
Nov 25 16:28:47.569277603 (392h24m46.120748054s) [seg_pos={0 0}, upload=345561]: Done begin upload segment.
Nov 25 16:28:47.569301709 (392h24m46.12077217s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569345382 (392h24m46.120815833s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569941547 (392h24m46.121412008s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570293928 (392h24m46.121764379s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570788309 (392h24m46.12225877s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570913107 (392h24m46.122383578s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.571412828 (392h24m46.122883279s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572270512 (392h24m46.123740973s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572772387 (392h24m46.124242838s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.573920553 (392h24m46.125391014s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574282523 (392h24m46.125752984s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574774430 (392h24m46.126244881s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.575325579 (392h24m46.12679603s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576162292 (392h24m46.127632774s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576973507 (392h24m46.128443968s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.578218949 (392h24m46.12968941s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:49.407103784 (392h24m47.958574245s) [upload=345561]: Next returned nil segment
Nov 25 16:28:49.407229494 (392h24m47.958699955s) [upload=345561]: Waiting for error group managing segments...
Nov 25 16:28:49.883090241 (392h24m48.434560692s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.082752298 (392h24m48.634222749s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.125258639 (392h24m48.676729101s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:51.076449024 (392h24m49.627919476s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:53.354542190 (392h24m51.906012641s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:55.609075859 (392h24m54.160546321s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:05.493697001 (392h25m4.045167462s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:08.267247120 (392h25m6.818717581s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:10.168294941 (392h25m8.719765402s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:11.208620898 (392h25m9.760091359s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:14.440065197 (392h25m12.991535668s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:15.559910707 (392h25m14.111381168s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:16.627776183 (392h25m15.179246644s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection
Nov 25 16:29:16.627908315 (392h25m15.179378776s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection
Nov 25 16:29:16.627972317 (392h25m15.179442778s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection
Nov 25 16:29:16.628074692 (392h25m15.179545153s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection
Nov 25 16:29:16.628111291 (392h25m15.179581752s) [seg_pos={0 0}, upload=345561]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 25 16:29:16.870655766 (392h25m15.422126227s) [upload=345561]: Done uploading segments...
Nov 25 16:29:16.870701794 (392h25m15.422172265s) [upload=345561]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.989172464s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
701.643361ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:15:55 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 701.677746ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [XX..........................................................]
duration:
5.85031923s
trace:
https://jaeger.dev.storj.io/trace/55ff491f49891b49
FAIL (2025-11-25 17:15:23 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 25 16:28:47.321164002 (392h24m45.872634453s) [upload=345561]: Starting upload
Nov 25 16:28:47.321244084 (392h24m45.872714535s) [upload=345561]: Done starting upload
Nov 25 16:28:47.321342281 (392h24m45.872812742s) [upload=345561]: Uploading segments...
Nov 25 16:28:47.321480945 (392h24m45.872951396s) [upload=345561]: Got next segment. Inline: false
Nov 25 16:28:47.321521653 (392h24m45.872992114s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 25 16:28:47.321528445 (392h24m45.872998897s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 25 16:28:47.569226625 (392h24m46.120697086s) [seg_pos={0 0}, upload=345561]: Begin upload segment...
Nov 25 16:28:47.569277603 (392h24m46.120748054s) [seg_pos={0 0}, upload=345561]: Done begin upload segment.
Nov 25 16:28:47.569301709 (392h24m46.12077217s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569345382 (392h24m46.120815833s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569941547 (392h24m46.121412008s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570293928 (392h24m46.121764379s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570788309 (392h24m46.12225877s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570913107 (392h24m46.122383578s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.571412828 (392h24m46.122883279s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572270512 (392h24m46.123740973s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572772387 (392h24m46.124242838s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.573920553 (392h24m46.125391014s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574282523 (392h24m46.125752984s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574774430 (392h24m46.126244881s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.575325579 (392h24m46.12679603s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576162292 (392h24m46.127632774s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576973507 (392h24m46.128443968s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.578218949 (392h24m46.12968941s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:49.407103784 (392h24m47.958574245s) [upload=345561]: Next returned nil segment
Nov 25 16:28:49.407229494 (392h24m47.958699955s) [upload=345561]: Waiting for error group managing segments...
Nov 25 16:28:49.883090241 (392h24m48.434560692s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.082752298 (392h24m48.634222749s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.125258639 (392h24m48.676729101s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:51.076449024 (392h24m49.627919476s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:53.354542190 (392h24m51.906012641s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:55.609075859 (392h24m54.160546321s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:05.493697001 (392h25m4.045167462s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:08.267247120 (392h25m6.818717581s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:10.168294941 (392h25m8.719765402s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:11.208620898 (392h25m9.760091359s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:14.440065197 (392h25m12.991535668s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:15.559910707 (392h25m14.111381168s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:16.627776183 (392h25m15.179246644s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection
Nov 25 16:29:16.627908315 (392h25m15.179378776s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection
Nov 25 16:29:16.627972317 (392h25m15.179442778s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection
Nov 25 16:29:16.628074692 (392h25m15.179545153s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection
Nov 25 16:29:16.628111291 (392h25m15.179581752s) [seg_pos={0 0}, upload=345561]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 25 16:29:16.870655766 (392h25m15.422126227s) [upload=345561]: Done uploading segments...
Nov 25 16:29:16.870701794 (392h25m15.422172265s) [upload=345561]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.850356451s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
699.068678ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:14:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 699.126087ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [.XX.........................................................]
duration:
5.819417995s
trace:
https://jaeger.dev.storj.io/trace/55ff491f49891b49
FAIL (2025-11-25 17:14:12 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 25 16:28:47.321164002 (392h24m45.872634453s) [upload=345561]: Starting upload
Nov 25 16:28:47.321244084 (392h24m45.872714535s) [upload=345561]: Done starting upload
Nov 25 16:28:47.321342281 (392h24m45.872812742s) [upload=345561]: Uploading segments...
Nov 25 16:28:47.321480945 (392h24m45.872951396s) [upload=345561]: Got next segment. Inline: false
Nov 25 16:28:47.321521653 (392h24m45.872992114s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 25 16:28:47.321528445 (392h24m45.872998897s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 25 16:28:47.569226625 (392h24m46.120697086s) [seg_pos={0 0}, upload=345561]: Begin upload segment...
Nov 25 16:28:47.569277603 (392h24m46.120748054s) [seg_pos={0 0}, upload=345561]: Done begin upload segment.
Nov 25 16:28:47.569301709 (392h24m46.12077217s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569345382 (392h24m46.120815833s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569941547 (392h24m46.121412008s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570293928 (392h24m46.121764379s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570788309 (392h24m46.12225877s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570913107 (392h24m46.122383578s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.571412828 (392h24m46.122883279s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572270512 (392h24m46.123740973s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572772387 (392h24m46.124242838s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.573920553 (392h24m46.125391014s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574282523 (392h24m46.125752984s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574774430 (392h24m46.126244881s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.575325579 (392h24m46.12679603s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576162292 (392h24m46.127632774s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576973507 (392h24m46.128443968s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.578218949 (392h24m46.12968941s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:49.407103784 (392h24m47.958574245s) [upload=345561]: Next returned nil segment
Nov 25 16:28:49.407229494 (392h24m47.958699955s) [upload=345561]: Waiting for error group managing segments...
Nov 25 16:28:49.883090241 (392h24m48.434560692s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.082752298 (392h24m48.634222749s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.125258639 (392h24m48.676729101s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:51.076449024 (392h24m49.627919476s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:53.354542190 (392h24m51.906012641s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:55.609075859 (392h24m54.160546321s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:05.493697001 (392h25m4.045167462s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:08.267247120 (392h25m6.818717581s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:10.168294941 (392h25m8.719765402s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:11.208620898 (392h25m9.760091359s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:14.440065197 (392h25m12.991535668s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:15.559910707 (392h25m14.111381168s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:16.627776183 (392h25m15.179246644s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection
Nov 25 16:29:16.627908315 (392h25m15.179378776s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection
Nov 25 16:29:16.627972317 (392h25m15.179442778s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection
Nov 25 16:29:16.628074692 (392h25m15.179545153s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection
Nov 25 16:29:16.628111291 (392h25m15.179581752s) [seg_pos={0 0}, upload=345561]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 25 16:29:16.870655766 (392h25m15.422126227s) [upload=345561]: Done uploading segments...
Nov 25 16:29:16.870701794 (392h25m15.422172265s) [upload=345561]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.819440529s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
699.444574ms
target:
https://us1.storj.io/login
trace:
https://jaeger.dev.storj.io/trace/290d15fae21b1e85
FAIL (2025-11-25 17:14:06 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused
Duration: 699.462458ms core satellite us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
2 out of 60 is failed (threshold=3) [..XX........................................................]
duration:
5.840027501s
trace:
https://jaeger.dev.storj.io/trace/55ff491f49891b49
FAIL (2025-11-25 17:13:20 UTC)
orbiter warning: 
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Nov 25 16:28:47.321164002 (392h24m45.872634453s) [upload=345561]: Starting upload
Nov 25 16:28:47.321244084 (392h24m45.872714535s) [upload=345561]: Done starting upload
Nov 25 16:28:47.321342281 (392h24m45.872812742s) [upload=345561]: Uploading segments...
Nov 25 16:28:47.321480945 (392h24m45.872951396s) [upload=345561]: Got next segment. Inline: false
Nov 25 16:28:47.321521653 (392h24m45.872992114s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Nov 25 16:28:47.321528445 (392h24m45.872998897s) [upload=345561]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Nov 25 16:28:47.569226625 (392h24m46.120697086s) [seg_pos={0 0}, upload=345561]: Begin upload segment...
Nov 25 16:28:47.569277603 (392h24m46.120748054s) [seg_pos={0 0}, upload=345561]: Done begin upload segment.
Nov 25 16:28:47.569301709 (392h24m46.12077217s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569345382 (392h24m46.120815833s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.569941547 (392h24m46.121412008s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570293928 (392h24m46.121764379s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570788309 (392h24m46.12225877s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.570913107 (392h24m46.122383578s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.571412828 (392h24m46.122883279s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572270512 (392h24m46.123740973s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.572772387 (392h24m46.124242838s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.573920553 (392h24m46.125391014s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574282523 (392h24m46.125752984s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.574774430 (392h24m46.126244881s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.575325579 (392h24m46.12679603s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576162292 (392h24m46.127632774s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.576973507 (392h24m46.128443968s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:47.578218949 (392h24m46.12968941s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Uploading piece...
Nov 25 16:28:49.407103784 (392h24m47.958574245s) [upload=345561]: Next returned nil segment
Nov 25 16:28:49.407229494 (392h24m47.958699955s) [upload=345561]: Waiting for error group managing segments...
Nov 25 16:28:49.883090241 (392h24m48.434560692s) [noise=true , address=77.90.31.12:4590     , piece_id=QFHXJWSCVA6OTXJMMATFJY3F3IC2F73VMRSBEUAFIGW4H4ASDEKQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.082752298 (392h24m48.634222749s) [noise=true , address=145.79.227.9:4366    , piece_id=MO3KDHLLSO4Z3J4GP7WUE7RPTBBVTBNQG735B3JJVPDFFIBLJXDA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:50.125258639 (392h24m48.676729101s) [noise=true , address=145.79.227.28:4727   , piece_id=D7CPKVTZS5IXOVUTSFNGRTACTEW2CJXAEGNWHJKJ7QFPKPLZFJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:51.076449024 (392h24m49.627919476s) [noise=true , address=145.79.227.34:4906   , piece_id=OPT3Y2OOSZVCXKOIFM5DS52X5DWFAPNITHRDRMYHMFCTWIT5WYKA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:53.354542190 (392h24m51.906012641s) [noise=true , address=145.79.227.14:4837   , piece_id=7WMZSQAWRJ5NLWJW4P6OGN66F3ULITOPPGFIJDN3KV5DOVZZHGFA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:28:55.609075859 (392h24m54.160546321s) [noise=true , address=145.79.227.40:4051   , piece_id=YL7ZITSRXBQS74QPNLOAITBGZDDSYOWKA4FHC6CBBZ7GSVKKLCCQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:05.493697001 (392h25m4.045167462s) [noise=true , address=145.79.227.19:4078   , piece_id=NE32VKDRLOJZM4NFT5WEZLJNYC52XTBHCNUZAQJNWBN2GASWWXYQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:08.267247120 (392h25m6.818717581s) [noise=true , address=145.79.227.42:4123   , piece_id=3ULFN7YVKFNAOKXXEPYYDCDWRPI2HF6A6HEMK5ZI4JMP6UEFF2TA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:10.168294941 (392h25m8.719765402s) [noise=true , address=145.79.227.23:4547   , piece_id=AAEFH4MGHLRDMRFHQ22R3PDY32FHTQRGKLDSZHKPBXH5Y476QTHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:11.208620898 (392h25m9.760091359s) [noise=true , address=145.79.227.25:4618   , piece_id=VY4I2FFCMPQ7C63OPYDMAMIU577API5E5CKKVHHSND4WTK2ADB5A, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:14.440065197 (392h25m12.991535668s) [noise=true , address=145.79.227.24:4582   , piece_id=CEWTJCCUNLWWTFP3WD2PK75AJSFAN65ADXTNVNHYFGF7MRATZRUA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:15.559910707 (392h25m14.111381168s) [noise=true , address=145.79.227.18:4006   , piece_id=DILD5MCP2LWRNKMASK3JR4WNWB7HQ5SUYTBLGQ24A26JA4X7RBNA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: <nil>
Nov 25 16:29:16.627776183 (392h25m15.179246644s) [noise=true , address=145.79.227.13:4801   , piece_id=VNW643IY5QZV7CCSE5KLJBXES3R3PTAILRDVTOX6TQPF2WQ7JJRQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34574->145.79.227.13:4801: use of closed network connection
Nov 25 16:29:16.627908315 (392h25m15.179378776s) [noise=true , address=145.79.227.32:4836   , piece_id=ZMO6MBJ4PARVLE4K4JQZLVIZF2NADGWKYHCPCXMBPZQ6A234IENQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41258->145.79.227.32:4836: use of closed network connection
Nov 25 16:29:16.627972317 (392h25m15.179442778s) [noise=true , address=145.79.227.10:4402   , piece_id=SKZIJE7F3H4V2LAMR6IBXHHTV4ZDKTIB2FDMMNJL4F25BHNYSKHA, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55454->145.79.227.10:4402: use of closed network connection
Nov 25 16:29:16.628074692 (392h25m15.179545153s) [noise=true , address=145.79.227.6:4257    , piece_id=2SEYCRNQ3HS4BQLFRB4Y6T2GUZIPWFNZUDMAVKNTBYVXXROKBURQ, seg_pos={0 0}, upload=345561]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43084->145.79.227.6:4257: use of closed network connection
Nov 25 16:29:16.628111291 (392h25m15.179581752s) [seg_pos={0 0}, upload=345561]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Nov 25 16:29:16.870655766 (392h25m15.422126227s) [upload=345561]: Done uploading segments...
Nov 25 16:29:16.870701794 (392h25m15.422172265s) [upload=345561]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 5.840058609s core uplink us-select-1