Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................] duration:
1.418464009s
FAIL (2025-06-25 06:10:45 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.418487384s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.935701888s
FAIL (2025-06-25 06:10:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.935718318s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
5.865713706s
FAIL (2025-06-25 06:10:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.865729527s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................] duration:
1.307467614s
FAIL (2025-06-25 06:09:45 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.307490918s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.881992938s
FAIL (2025-06-25 06:09:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.882040581s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
5.731696523s
FAIL (2025-06-25 06:09:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.731718026s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................] duration:
1.3131127s
FAIL (2025-06-25 06:09:06 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.313136765s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
1.31070813s
FAIL (2025-06-25 06:08:50 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.310735654s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.796920418s
FAIL (2025-06-25 06:08:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.796934405s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
5.963055988s
FAIL (2025-06-25 06:08:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.963071077s
edge
gateway
us-select-1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.000872966s
FAIL (2025-06-25 06:08:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000903806s
core
uplink
us-select-1
us1-placement-20
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
1.318672759s
FAIL (2025-06-25 06:08:10 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.318701395s
edge
authservice
http
bgp
us1
|
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
30.001098108s
FAIL (2025-06-25 06:07:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001117315s
edge
gateway
us-select-1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
1.310412631s
FAIL (2025-06-25 06:07:27 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.310437249s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
1.315449355s
FAIL (2025-06-25 06:06:47 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.315464756s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
1.305531877s
FAIL (2025-06-25 06:05:56 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.305557668s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
1.362259511s
FAIL (2025-06-25 06:04:43 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.362288969s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
1.312262704s
FAIL (2025-06-25 06:04:14 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.312307981s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
1.358761112s
FAIL (2025-06-25 06:03:44 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.358799848s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
1.311039375s
FAIL (2025-06-25 06:03:00 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.311089072s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
1.292038264s
FAIL (2025-06-25 06:01:48 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.292079663s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
1.31997465s
FAIL (2025-06-25 06:01:24 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.32000001s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
1.34231552s
FAIL (2025-06-25 06:00:30 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.342344627s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
1.343287352s
FAIL (2025-06-25 05:59:19 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.343315496s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
1.384883324s
FAIL (2025-06-25 05:58:05 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.38490147s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
1.34444245s
FAIL (2025-06-25 05:57:18 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.344468149s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
1.399554729s
FAIL (2025-06-25 05:56:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.399584186s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
1.354735335s
FAIL (2025-06-25 05:55:40 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.354767729s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
1.317501116s
FAIL (2025-06-25 05:54:46 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.317524702s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
1.339276093s
FAIL (2025-06-25 05:53:47 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.339295841s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
1.313388766s
FAIL (2025-06-25 05:52:39 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.313413473s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
1.385703142s
FAIL (2025-06-25 05:51:56 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.385746877s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
1.306795205s
FAIL (2025-06-25 05:50:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.306823359s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
1.349690879s
FAIL (2025-06-25 05:50:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.349732421s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
1.323916971s
FAIL (2025-06-25 05:49:39 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.323934134s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
1.361472579s
FAIL (2025-06-25 05:49:07 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1.361486606s
edge
authservice
http
bgp
us1
|
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m55.725011186s
FAIL (2025-06-25 05:49:05 UTC)
orbiter warning: unexpected status code "500 Internal Server Error" "\n"
Duration: 1m55.725030143s
edge
authservice
http
bgp
us1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
5.813111023s
FAIL (2025-06-25 02:41:49 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 5.81314053s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
8.098525764s
FAIL (2025-06-25 02:41:08 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.098549009s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
8.766128955s
FAIL (2025-06-25 02:40:24 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.76615265s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
7.155183931s
FAIL (2025-06-25 02:39:24 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 7.155205133s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
8.483587423s
FAIL (2025-06-25 02:38:51 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.48361267s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
8.116079683s
FAIL (2025-06-25 02:38:33 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.116104261s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
8.035382923s
FAIL (2025-06-25 02:38:14 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.035443941s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
8.46470555s
FAIL (2025-06-25 02:37:16 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.464730959s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
17.003899303s
FAIL (2025-06-25 02:36:18 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 17.003926887s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.904955232s
FAIL (2025-06-25 02:35:26 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 6.904981624s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
13.395973195s
FAIL (2025-06-25 02:34:38 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 13.396000819s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
7.413760073s
FAIL (2025-06-25 02:33:53 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 7.413785973s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................] duration:
14.996728071s
FAIL (2025-06-25 02:33:22 UTC)
orbiter warning:
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009 , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973 , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734 , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553 , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628 , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519 , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083 , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227 , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225 , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154 , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119 , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518 , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257 , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838 , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663 , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221 , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 14.996752649s
core
uplink
us-select-1
|