Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 433427 reports (Page 8618 of 8669)
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...............................X............................]
duration:
1.418464009s
FAIL (2025-06-25 06:10:45 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.418487384s edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
5.935701888s
FAIL (2025-06-25 06:10:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.935718318s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
5.865713706s
FAIL (2025-06-25 06:10:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.865729527s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [................................X...........................]
duration:
1.307467614s
FAIL (2025-06-25 06:09:45 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.307490918s edge authservice http bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
5.881992938s
FAIL (2025-06-25 06:09:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.882040581s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
5.731696523s
FAIL (2025-06-25 06:09:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.731718026s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.................................X..........................]
duration:
1.3131127s
FAIL (2025-06-25 06:09:06 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.313136765s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................]
duration:
1.31070813s
FAIL (2025-06-25 06:08:50 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.310735654s edge authservice http bgp us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
5.796920418s
FAIL (2025-06-25 06:08:22 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.796934405s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
5.963055988s
FAIL (2025-06-25 06:08:19 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.963071077s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
1m0.000872966s
FAIL (2025-06-25 06:08:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000903806s core uplink us-select-1 us1-placement-20
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................]
duration:
1.318672759s
FAIL (2025-06-25 06:08:10 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.318701395s edge authservice http bgp us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
30.001098108s
FAIL (2025-06-25 06:07:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 30.001117315s edge gateway us-select-1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................]
duration:
1.310412631s
FAIL (2025-06-25 06:07:27 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.310437249s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................]
duration:
1.315449355s
FAIL (2025-06-25 06:06:47 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.315464756s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................]
duration:
1.305531877s
FAIL (2025-06-25 06:05:56 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.305557668s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................]
duration:
1.362259511s
FAIL (2025-06-25 06:04:43 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.362288969s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................]
duration:
1.312262704s
FAIL (2025-06-25 06:04:14 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.312307981s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................]
duration:
1.358761112s
FAIL (2025-06-25 06:03:44 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.358799848s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................]
duration:
1.311039375s
FAIL (2025-06-25 06:03:00 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.311089072s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................]
duration:
1.292038264s
FAIL (2025-06-25 06:01:48 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.292079663s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............]
duration:
1.31997465s
FAIL (2025-06-25 06:01:24 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.32000001s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............]
duration:
1.34231552s
FAIL (2025-06-25 06:00:30 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.342344627s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............]
duration:
1.343287352s
FAIL (2025-06-25 05:59:19 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.343315496s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............]
duration:
1.384883324s
FAIL (2025-06-25 05:58:05 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.38490147s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........]
duration:
1.34444245s
FAIL (2025-06-25 05:57:18 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.344468149s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........]
duration:
1.399554729s
FAIL (2025-06-25 05:56:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.399584186s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........]
duration:
1.354735335s
FAIL (2025-06-25 05:55:40 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.354767729s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........]
duration:
1.317501116s
FAIL (2025-06-25 05:54:46 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.317524702s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......]
duration:
1.339276093s
FAIL (2025-06-25 05:53:47 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.339295841s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......]
duration:
1.313388766s
FAIL (2025-06-25 05:52:39 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.313413473s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....]
duration:
1.385703142s
FAIL (2025-06-25 05:51:56 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.385746877s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....]
duration:
1.306795205s
FAIL (2025-06-25 05:50:54 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.306823359s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...]
duration:
1.349690879s
FAIL (2025-06-25 05:50:09 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.349732421s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..]
duration:
1.323916971s
FAIL (2025-06-25 05:49:39 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.323934134s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.]
duration:
1.361472579s
FAIL (2025-06-25 05:49:07 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1.361486606s edge authservice http bgp us1
checkers.AuthServiceHTTP{"https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777"} failure (antiblip window 3/60): (authservice.us1)
antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X]
duration:
1m55.725011186s
FAIL (2025-06-25 05:49:05 UTC)
orbiter warning: unexpected status code "500 Internal Server Error"
"\n"
Duration: 1m55.725030143s edge authservice http bgp us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................]
duration:
5.813111023s
FAIL (2025-06-25 02:41:49 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 5.81314053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration:
8.098525764s
FAIL (2025-06-25 02:41:08 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.098549009s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration:
8.766128955s
FAIL (2025-06-25 02:40:24 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.76615265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................]
duration:
7.155183931s
FAIL (2025-06-25 02:39:24 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 7.155205133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration:
8.483587423s
FAIL (2025-06-25 02:38:51 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.48361267s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration:
8.116079683s
FAIL (2025-06-25 02:38:33 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.116104261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration:
8.035382923s
FAIL (2025-06-25 02:38:14 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.035443941s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration:
8.46470555s
FAIL (2025-06-25 02:37:16 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 8.464730959s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................]
duration:
17.003899303s
FAIL (2025-06-25 02:36:18 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 17.003926887s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................]
duration:
6.904955232s
FAIL (2025-06-25 02:35:26 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 6.904981624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................]
duration:
13.395973195s
FAIL (2025-06-25 02:34:38 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 13.396000819s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................]
duration:
7.413760073s
FAIL (2025-06-25 02:33:53 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 7.413785973s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip:
1 out of 60 is failed (threshold=3) [............X...............................................]
duration:
14.996728071s
FAIL (2025-06-25 02:33:22 UTC)
orbiter warning: 
======ERROR======
uplink: context deadline exceeded
======LOGS======
tchRequestItem_SegmentCommit
Jun 25 01:56:51.221178176 (40h46m25.991819511s) [upload=32531]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:56:51.515131203 (40h46m26.285772528s) [upload=32531]: Done uploading segments...
Jun 25 01:56:51.515147645 (40h46m26.28578898s) [upload=32531]: Upload finished. err: <nil>
Jun 25 01:57:19.084236458 (40h46m53.854877823s) [upload=32538]: Starting upload
Jun 25 01:57:19.084344367 (40h46m53.854985722s) [upload=32538]: Done starting upload
Jun 25 01:57:19.084468798 (40h46m53.855110173s) [upload=32538]: Uploading segments...
Jun 25 01:57:19.084697521 (40h46m53.855338876s) [upload=32538]: Got next segment. Inline: false
Jun 25 01:57:19.084747819 (40h46m53.855389174s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Jun 25 01:57:19.084755283 (40h46m53.855396648s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Jun 25 01:57:19.317920137 (40h46m54.088561472s) [seg_pos={0 0}, upload=32538]: Begin upload segment...
Jun 25 01:57:19.317968261 (40h46m54.088609596s) [seg_pos={0 0}, upload=32538]: Done begin upload segment.
Jun 25 01:57:19.317996586 (40h46m54.088637921s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318047705 (40h46m54.08868903s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318346855 (40h46m54.08898818s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318526723 (40h46m54.089168058s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318602399 (40h46m54.089243734s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318842113 (40h46m54.089483448s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.318925244 (40h46m54.089566589s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319092588 (40h46m54.089733923s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319330209 (40h46m54.089971534s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319576726 (40h46m54.090218071s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319829686 (40h46m54.090471021s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.319884112 (40h46m54.090525447s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320118615 (40h46m54.09075995s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320332750 (40h46m54.090974085s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320593695 (40h46m54.09123502s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:19.320672939 (40h46m54.091314274s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Uploading piece...
Jun 25 01:57:21.004186623 (40h46m55.774827958s) [upload=32538]: Next returned nil segment
Jun 25 01:57:21.004201271 (40h46m55.774842606s) [upload=32538]: Waiting for error group managing segments...
Jun 25 01:57:21.373405862 (40h46m56.144047197s) [noise=true , address=45.91.116.7:4009     , piece_id=HQLHWOX2J3WRLZMMMT4U3G74SEQ6JXEUUW6LPMTRIAYFKSA7VYDQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.380540770 (40h46m56.151182105s) [noise=true , address=45.91.116.6:4973     , piece_id=NB4TVDRSLPU5MJNSP2S46X4Y7GQZCI4ZFGE636VNMZJVFCYPND2Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406216388 (40h46m56.176857723s) [noise=true , address=77.90.31.16:4734     , piece_id=FV3BW4KKGILZ64MDYMVZCWBTXLHGJ42KT6OPRQ3D5PNFHC2MXK4A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.406531558 (40h46m56.177172893s) [noise=true , address=77.90.31.11:4553     , piece_id=LY6ZNU54JOBMSQUHKIVRSIO4DHRDINUYYAJEZDCJG2Y2JN3ZTH2A, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.413930989 (40h46m56.184572324s) [noise=true , address=77.90.31.13:4628     , piece_id=V3NBPYML5LGRAH4VWIDC5MNEXKBTIB22I2ICYLESNAEIZWRS4JSA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.418444851 (40h46m56.189086186s) [noise=true , address=77.90.31.10:4519     , piece_id=BZY2WT4HER4FFFSQFKNPEDUL6DAXOLMCTPC6BBBYGAMGIHAAM4UQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.450429187 (40h46m56.221070512s) [noise=true , address=45.91.116.10:4083    , piece_id=ZXYVYU6RYXS2DUJY5SFRHIAQ5C5A7VFR5WUXHY55PFSZ5C5ION5Q, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.451791586 (40h46m56.222432921s) [noise=true , address=45.91.116.12:5227    , piece_id=WAYLVFZ5JZGAA2DO2TFCQO5HQN5FKTED642GKLXNPXJH5426L2RQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.461695047 (40h46m56.232336372s) [noise=true , address=45.91.116.12:5225    , piece_id=ZYBO4RY72O2ENHQL43DMR7PJWKSVIXOUB4MFGTPVRB7VDYKLVSGQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.465101795 (40h46m56.23574312s) [noise=true , address=45.91.116.13:4154    , piece_id=5CO7RVYGMY66M5NGEQ2W3MG6WFHEGMWSOVP24HYFDANQH6LLGWUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:21.516253534 (40h46m56.286894869s) [noise=true , address=45.91.116.11:4119    , piece_id=LJOG5IILWQCG2QAQ7JBEBYW7T5ZAFKOMEU2735VAUPZMEVUV4CMA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:22.126274879 (40h46m56.896916214s) [noise=true , address=77.90.31.10:4518     , piece_id=KXNW7PNSTLKXHXRFVXSPB63YIOHVDQREL3D3GLYQJDTKOFMTMDUQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256195959 (40h47m4.026837284s) [noise=true , address=104.36.82.15:4257    , piece_id=N5IMTSWIGPENDENU4XGCG4FXZKGUOT2GWA6KCJYVEAKKMVSNC6SQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: <nil>
Jun 25 01:57:29.256217671 (40h47m4.026859006s) [seg_pos={0 0}, upload=32538]: Segment reached optimal threshold of 13 pieces.
Jun 25 01:57:29.256444921 (40h47m4.027086246s) [noise=true , address=104.36.82.5:4838     , piece_id=UXOJRBK2YWWMLUMBOCWEWDEI4QH56AVCQG3WRIT2UKQ67XRJ6VWQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36982->104.36.82.5:4838: use of closed network connection
Jun 25 01:57:29.256487343 (40h47m4.027128678s) [noise=true , address=77.90.31.14:4663     , piece_id=OBYHC3FUIR6W6MVC7CKF25SW5JQZCYPPII2H4MOI3D6J4SG5K6VQ, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:155F5vnBUAgJYDuEgxwgnHCEVarkUAcdNjqh7V52vx65uHyJfE): protocol: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54040->77.90.31.14:4663: use of closed network connection
Jun 25 01:57:29.256544754 (40h47m4.027186079s) [noise=true , address=104.36.82.14:4221    , piece_id=7RB7VEGUDNT2ZBNJWHEMHF4YEL7PHDAY4XYO47P2DWRJPOJEYIAA, seg_pos={0 0}, upload=32538]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR): protocol: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50190->104.36.82.14:4221: use of closed network connection
Jun 25 01:57:29.256578479 (40h47m4.027219814s) [seg_pos={0 0}, upload=32538]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Jun 25 01:57:29.256660598 (40h47m4.027301923s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Jun 25 01:57:29.256667341 (40h47m4.027308676s) [upload=32538]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Jun 25 01:57:29.697233572 (40h47m4.467874937s) [upload=32538]: Done uploading segments...
Jun 25 01:57:29.697257679 (40h47m4.467899055s) [upload=32538]: Upload finished. err: <nil>
Duration: 14.996752649s core uplink us-select-1