Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..X.X.........X........XX........................X..........] duration: 5.752573072s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:55:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.752599541s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 2.011142331s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:55:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.011177117s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...X.X.........X........XX........................X.........] duration: 5.947658993s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:55:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.947685553s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 477.636993ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:55:16 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 477.662221ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.........X.....X.........................X..................] duration: 3.441034279s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:55:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.441489576s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.174910372s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:54:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.175063675s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 2.368988427s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:54:53 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.369019036s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [....X.X.........X........XX........................X........] duration: 5.935036467s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:54:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.935054581s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..........X.....X.........................X.................] duration: 3.479238856s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:54:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.479265646s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 473.579386ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:54:30 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 473.609563ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.612675567s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:54:18 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.612708619s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.937911468s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:54:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.937948548s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 474.03844ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:53:55 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 474.067966ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...........X.....X.........................X................] duration: 3.945070528s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:53:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.945096658s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 2.01784988s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:53:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.017877703s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.....X.X.........X........XX........................X.......] duration: 5.807163308s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:53:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.807260161s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.071909529s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:53:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.071939344s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 485.750552ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:53:23 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 485.782552ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 7.546175902s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:53:21 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 7.546199977s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 2.77034682s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:53:19 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.770375254s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [............X.....X.........................X...............] duration: 3.429511276s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:53:12 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.429538137s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [......X.X.........X........XX........................X......] duration: 5.711576727s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:53:05 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.711737474s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 16.622513257s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:53:01 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 16.622539869s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 7.027473705s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:52:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.027507889s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.......X.X.........X........XX........................X.....] duration: 5.739264969s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:52:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.739289486s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.826011054s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:52:26 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.826048975s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [.............X.....X.........................X..............] duration: 3.539868008s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:52:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.539888678s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 2.077932582s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:52:24 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.077950376s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 477.187256ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:52:18 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 477.211152ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [........X.X.........X........XX........................X....] duration: 5.825132378s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:51:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.825161664s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [..............X.....X.........................X.............] duration: 3.381439548s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:51:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.381469215s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 7.209186734s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:51:42 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 7.209215911s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.936565126s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:51:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.9365834s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 2.045812388s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:51:31 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.045845001s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.83617717s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:51:23 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.836192598s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 484.761199ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:51:21 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 484.789873ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.15068455s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:51:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.150710579s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [.........X.X.........X........XX........................X...] duration: 5.682010486s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:51:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.682039712s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [..........X.X.........X........XX........................X..] duration: 5.904894121s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:50:38 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.904939579s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.694006862s trace: https://jaeger.dev.storj.io/trace/4939b1d48dfb8f81 FAIL (2025-12-08 10:50:36 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.694040888s core uplink us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [...............X.....X.........................X............] duration: 3.553425452s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:50:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.553454057s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 2.022966195s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:50:27 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.02299515s core uplink us1
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 1.497925057s trace: https://jaeger.dev.storj.io/trace/3d2642fe33f63575 FAIL (2025-12-08 10:50:15 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 1.497955915s core uplink eu1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 469.380331ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:50:15 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 469.423684ms core satellite eu1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.934294543s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:50:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.934322135s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 2.061017562s trace: https://jaeger.dev.storj.io/trace/2e2423e2c55feee9 FAIL (2025-12-08 10:50:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 2.061050055s core uplink us1
checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 466.579376ms remote: "tcp:35.207.121.91:443" target: https://eu1.storj.io/login trace: https://jaeger.dev.storj.io/trace/9b67b097dee132a FAIL (2025-12-08 10:49:53 UTC) orbiter warning: Get "https://eu1.storj.io/login": EOF Duration: 466.629612ms core satellite eu1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 3 out of 60 is failed (threshold=6) [................X.....X.........................X...........] duration: 3.353533801s trace: https://jaeger.dev.storj.io/trace/1d0975ff047972b9 FAIL (2025-12-08 10:49:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.353591701s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 6 out of 60 is failed (threshold=3) [...........X.X.........X........XX........................X.] duration: 5.795663862s trace: https://jaeger.dev.storj.io/trace/6d728abedff5f55a FAIL (2025-12-08 10:49:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.795690053s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.97176067s trace: https://jaeger.dev.storj.io/trace/6bb63dcf18ba7880 FAIL (2025-12-08 10:49:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== h44m16.983490049s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.112992650 (66h44m16.984075695s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113418911 (66h44m16.984501956s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.113929985 (66h44m16.98501303s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114383147 (66h44m16.985466183s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.114899351 (66h44m16.985982386s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.115393962 (66h44m16.986477007s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.116760534 (66h44m16.987843579s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.118231125 (66h44m16.98931416s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.119778110 (66h44m16.990861156s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.121927263 (66h44m16.993010308s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.124202706 (66h44m16.995285751s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.126735178 (66h44m16.997818213s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.128218984 (66h44m16.999302019s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:17.129359496 (66h44m17.000442541s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Uploading piece... Dec 8 10:13:18.574550811 (66h44m18.445633856s) [upload=58503]: Next returned nil segment Dec 8 10:13:18.574570768 (66h44m18.445653813s) [upload=58503]: Waiting for error group managing segments... Dec 8 10:13:18.961005037 (66h44m18.832088092s) [noise=true , address=5.231.73.2:4001 , piece_id=VGN3S7FOQ6PMLJU27DJMJUM464NYMN5WD2ARFI2LORFVALEIZRAA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:18.987868641 (66h44m18.858951676s) [noise=true , address=145.79.227.43:4173 , piece_id=EBJTVGC2HXJDMH24ORDVDTLJ2VVO4Y3W4RRQNRE2CMEFUCJ2MPSQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.039287472 (66h44m18.910370517s) [noise=true , address=145.79.227.24:4596 , piece_id=CMXNJSJTJNUJ7OJGEFCPUJTYAJBUCFCLXO74WMQADOZAELHNEWJQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.061246838 (66h44m18.932329883s) [noise=true , address=5.231.73.4:4036 , piece_id=IQMPMGREE7J6MOQHL2ONXVGOYWAHDTXXSSUQU75VOF3CHY23UZ7Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.098402800 (66h44m18.969485845s) [noise=true , address=77.90.31.18:4827 , piece_id=PEXOCJPLLDBRLSZ3H5MQWLMVMCXC5ZIDX3TRZRMKMY2QE7XKO4ZQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.100826505 (66h44m18.97190955s) [noise=true , address=77.90.31.19:4868 , piece_id=UOYJ6R5QQ3NYYRDXLBFLS7KLX3GNJKF4UX4DF7SY4QFS6CXMKCKQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.101134542 (66h44m18.972217587s) [noise=true , address=77.90.31.17:4800 , piece_id=XIC5XF7CCOX5RYHIOC2QG526I4POMHHYPSDXPFEQUECGREYUIGXA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.165632308 (66h44m19.036715363s) [noise=true , address=45.91.116.8:4057 , piece_id=5UOHPLONR4MPPTLIS573LNEKHGTFELGYAVUQ54IYIQPS7KG4II2A, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.168379579 (66h44m19.039462625s) [noise=true , address=77.90.31.21:4886 , piece_id=BQONW4FVXHV6JJJC3XZCODQTZ5TI7GGOBJABM76ZB4TN5CES2PQQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169085244 (66h44m19.040168289s) [noise=true , address=77.90.31.12:4598 , piece_id=HPU6ZYDGZVYKQLFGTUQFFAEC3KKB3IN2QXYKCD4LJ4JGM3EP5HYQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.169511405 (66h44m19.04059445s) [noise=true , address=45.91.116.6:4993 , piece_id=CGOM3JR6Z3MQHZY6GDYVOCPGXKOYCRQ7UDRKYSY6VTLXPDT4RJXQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.182109444 (66h44m19.053192489s) [noise=true , address=77.90.31.11:4577 , piece_id=WS7U6QAXTCBCZMYVRJJHW4QVHHAYWHRFGECQ4UE6SK7O4RBJCRAQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.197552220 (66h44m19.068635255s) [noise=true , address=45.91.116.10:4112 , piece_id=KEC6NIM6J4JNL4RWX4OXVBEDFJGEKJIIHCR6OQF5TOSVD6WAN33Q, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213459772 (66h44m19.084542817s) [noise=true , address=45.91.116.12:4248 , piece_id=TXP4E7H4KINXK3WEOH47NORAGJZMIEZBY6O3KHZWPNYFACFGQTEA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: <nil> Dec 8 10:13:19.213485701 (66h44m19.084568746s) [seg_pos={0 0}, upload=58503]: Segment reached optimal threshold of 14 pieces. Dec 8 10:13:19.213729244 (66h44m19.084812289s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1hsbGK94RFwxic9F8NKp8kKvztegUhkNmYEor4QyLJmqF3seZf): protocol: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34274->64.49.13.169:4710: use of closed network connection Dec 8 10:13:19.213738002 (66h44m19.084821047s) [noise=true , address=64.49.13.169:4710 , piece_id=DOIR46GL4ORAUUK3Z262XP63SSCX2KJOIWTD5PENAZCWMSCAHIBA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.213843933 (66h44m19.084926978s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FgCn3deV8CMPtLrNN1oVxTwRE8c5RrsKEs9bna5MntLDM9Dzo): protocol: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51932->145.79.227.38:4002: use of closed network connection Dec 8 10:13:19.213860565 (66h44m19.08494361s) [noise=true , address=145.79.227.38:4002 , piece_id=5XQ2YBQMZ3FPWRG7UFGU6PKAOSQHZ2JFRGDZQWCR7HUIS5ZZ23IQ, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214078239 (66h44m19.085161274s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tJDCwfiqPgdc82nxvYrCiGB3KWijD4kSmYSAa8ZMBqRH2cA59): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214087327 (66h44m19.085170362s) [noise=true , address=145.79.227.30:4788 , piece_id=AC6TRAOAVXNGLMVQPMX3HWBHPBTN4S3XYDUADZWDK7NQUJQUCYWA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214227885 (66h44m19.08531093s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ftkFznfF2NSYrtgqUm3hgcyfwaZtWQfh2nDBEawxPh4SecemT): protocol: expected piece hash; context canceled; EOF Dec 8 10:13:19.214237402 (66h44m19.085320447s) [noise=true , address=77.90.31.16:4764 , piece_id=L6MPZIAQJQXFHMOVMCCPAT7NTEEY33EYT6CB7AUXAPGOGYMYUNPA, seg_pos={0 0}, upload=58503]: Long tail cancellation due to optimal threshold Dec 8 10:13:19.214271297 (66h44m19.085354332s) [seg_pos={0 0}, upload=58503]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Dec 8 10:13:19.214379413 (66h44m19.085462448s) [upload=58503]: Flush batch item: pb.BatchRequestItem_SegmentCommit Dec 8 10:13:19.214385644 (66h44m19.085468689s) [upload=58503]: Flush batch item: pb.BatchRequestItem_ObjectCommit Dec 8 10:15:27.040493313 (66h46m26.911576348s) [upload=58503]: Done uploading segments... Dec 8 10:15:27.040574868 (66h46m26.911657903s) [upload=58503]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.971786379s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.eu1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.HTTP{"GET" "https://eu1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.eu1.satellite_login)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)