Orbiter: EU/US/AP | Code | Config | Playbook | Edge Deployment | Escalation Procedure

Orbiter Check History

Showing 50 of 223249 reports (Page 1 of 4465)
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [X...........................................................]
duration: 
5.78026564s
FAIL (2025-10-27 16:26:56 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.780298664s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration: 
5.644870823s
FAIL (2025-10-27 16:26:23 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.644899458s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration: 
5.758175373s
FAIL (2025-10-27 16:25:41 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.758201864s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [...X........................................................]
duration: 
5.681921833s
FAIL (2025-10-27 16:25:21 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.681947962s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration: 
5.709353737s
FAIL (2025-10-27 16:25:01 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.709380388s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration: 
5.623045156s
FAIL (2025-10-27 16:23:49 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.623071557s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration: 
5.666752575s
FAIL (2025-10-27 16:23:11 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.666781719s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration: 
5.63724441s
FAIL (2025-10-27 16:22:05 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.637266671s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [........X...................................................]
duration: 
5.639253696s
FAIL (2025-10-27 16:21:45 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.639272231s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.........X..................................................]
duration: 
5.664081638s
FAIL (2025-10-27 16:21:07 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.664117396s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [..........X.................................................]
duration: 
5.777154983s
FAIL (2025-10-27 16:20:20 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.777186171s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [...........X................................................]
duration: 
5.786206292s
FAIL (2025-10-27 16:19:13 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.786232994s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [............X...............................................]
duration: 
5.699448956s
FAIL (2025-10-27 16:18:18 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.699472541s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.............X..............................................]
duration: 
5.583602695s
FAIL (2025-10-27 16:17:16 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.58371555s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [..............X.............................................]
duration: 
5.592887341s
FAIL (2025-10-27 16:16:09 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.592929001s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [X...........................................................]
duration: 
6.240783353s
FAIL (2025-10-27 16:15:57 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.240855149s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [...............X............................................]
duration: 
5.742278166s
FAIL (2025-10-27 16:15:47 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.742345153s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [X...........................................................]
duration: 
3.498744028s
FAIL (2025-10-27 16:15:37 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.498775898s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [................X...........................................]
duration: 
5.617938527s
FAIL (2025-10-27 16:15:31 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.617986849s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration: 
6.026780894s
FAIL (2025-10-27 16:15:26 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.026801203s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.................X..........................................]
duration: 
5.667089101s
FAIL (2025-10-27 16:15:02 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.667117966s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration: 
3.453034998s
FAIL (2025-10-27 16:14:42 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.453059534s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration: 
6.349761957s
FAIL (2025-10-27 16:14:30 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.34979405s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...X........................................................]
duration: 
6.160013181s
FAIL (2025-10-27 16:14:11 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.160042988s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [..................X.........................................]
duration: 
8.628326675s
FAIL (2025-10-27 16:13:56 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.628355099s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration: 
3.359636222s
FAIL (2025-10-27 16:13:38 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.359661981s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [...X........................................................]
duration: 
3.534141082s
FAIL (2025-10-27 16:13:15 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.534160469s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [...................X........................................]
duration: 
5.890085661s
FAIL (2025-10-27 16:13:06 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.890107482s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration: 
6.283943671s
FAIL (2025-10-27 16:12:58 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.283970731s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration: 
3.383947748s
FAIL (2025-10-27 16:12:15 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.383969088s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration: 
6.187682874s
FAIL (2025-10-27 16:12:13 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.187707111s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [X...........................................................]
duration: 
5.871375727s
FAIL (2025-10-27 16:12:10 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 5.871408509s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [....................X.......................................]
duration: 
5.699950204s
FAIL (2025-10-27 16:12:07 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.699977166s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.X..........................................................]
duration: 
5.959368043s
FAIL (2025-10-27 16:11:42 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 5.959472971s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration: 
3.424883483s
FAIL (2025-10-27 16:11:31 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.424908129s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.....................X......................................]
duration: 
5.662662566s
FAIL (2025-10-27 16:11:28 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.662686191s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration: 
6.175849158s
FAIL (2025-10-27 16:11:04 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.17587623s edge gateway us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [......X.....................................................]
duration: 
3.364584193s
FAIL (2025-10-27 16:11:02 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.364608679s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [..X.........................................................]
duration: 
6.039216291s
FAIL (2025-10-27 16:10:54 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 6.039258261s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [......................X.....................................]
duration: 
5.659728789s
FAIL (2025-10-27 16:10:50 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.659756593s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.......................X....................................]
duration: 
5.651203959s
FAIL (2025-10-27 16:10:34 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.651235199s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration: 
3.434125314s
FAIL (2025-10-27 16:10:15 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.4341537s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.......X....................................................]
duration: 
6.348077026s
FAIL (2025-10-27 16:10:14 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.348096402s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [...X........................................................]
duration: 
6.116263139s
FAIL (2025-10-27 16:10:11 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 6.116299217s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [........................X...................................]
duration: 
5.681013193s
FAIL (2025-10-27 16:10:00 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.681055834s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [....X.......................................................]
duration: 
6.192091847s
FAIL (2025-10-27 16:09:50 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 6.192124129s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 3/60): (gateway.us1.placement_20_thumbnails)
antiblip: 
1 out of 60 is failed (threshold=3) [........X...................................................]
duration: 
3.746070857s
FAIL (2025-10-27 16:09:38 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.746095655s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [.....X......................................................]
duration: 
5.898461565s
FAIL (2025-10-27 16:09:33 UTC) 
orbiter warning: 
operation: sync2.Copy
======ERROR======
uplink: piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: 76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ) context deadline exceeded; piecestore: (Node ID: 12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu, Piece ID: GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ) context deadline exceeded; piecestore: (Node ID: 12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, Piece ID: FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA) context deadline exceeded; piecestore: (Node ID: 1G3fe4SHMdNAjAmqLNyFb6qDKL94aAWGYE9qhczKmozKFhKKEV, Piece ID: 2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA) context deadline exceeded
======LOGS======
]: Done starting upload
Oct 27 15:28:22.900340512 (45m40.249647808s) [upload=668]: Uploading segments...
Oct 27 15:28:22.900564057 (45m40.249871343s) [upload=668]: Got next segment. Inline: false
Oct 27 15:28:22.900613051 (45m40.249920337s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Oct 27 15:28:22.900620896 (45m40.249928182s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Oct 27 15:28:23.560720563 (45m40.91002784s) [seg_pos={0 0}, upload=668]: Begin upload segment...
Oct 27 15:28:23.560802119 (45m40.910109405s) [seg_pos={0 0}, upload=668]: Done begin upload segment.
Oct 27 15:28:23.560839660 (45m40.910146947s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.560885057 (45m40.910192353s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561607883 (45m40.910915169s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561670783 (45m40.910978069s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.561966716 (45m40.911274002s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562189701 (45m40.911496987s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562277909 (45m40.911585195s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562572600 (45m40.911879887s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562586527 (45m40.911893813s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562888732 (45m40.912196018s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.562953846 (45m40.912261123s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563182882 (45m40.912490158s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563335423 (45m40.912642709s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563479036 (45m40.912786323s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.563711468 (45m40.913018754s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:23.564137260 (45m40.913444546s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Uploading piece...
Oct 27 15:28:27.241369967 (45m44.590677243s) [upload=668]: Next returned nil segment
Oct 27 15:28:27.241385446 (45m44.590692732s) [upload=668]: Waiting for error group managing segments...
Oct 27 15:28:29.744963349 (45m47.094270636s) [noise=true , address=145.79.227.5:4222    , piece_id=B7W6PCNLLAPHXL7XOSJXEOVX6XZGNFDJWXR7MVYDGCONQCN55HYA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:29.821202504 (45m47.17050978s) [noise=true , address=145.79.227.21:4475   , piece_id=VDOFRCJKDQLXV4CBM4MA3UGK2PZMRJ3BW6H7PKJRZMZXW56Y5G4A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:32.126653324 (45m49.47596061s) [noise=true , address=77.90.31.18:4807     , piece_id=OPDBUFUCTPZN2HH2SORHJL5L4EX2VWXATEFEIIEN5QTBQQFRGSZA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:33.087272849 (45m50.436580125s) [noise=true , address=45.91.116.5:4298     , piece_id=ATXRIPPIX6BAHUXCENJVDJ25K2Z6UH3QYL4MTBWPZKP7LFGHBX6A, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:38.581899084 (45m55.93120637s) [noise=true , address=145.79.227.26:4653   , piece_id=MXREE4NCFOGYEHVQNFHZYD5DLD4PHBJS64JYSLTXXJZG7AC3LH6Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.053322174 (45m56.40262946s) [noise=true , address=145.79.227.24:4582   , piece_id=K7COJQ2UG3CTGF4JVO27PTGWY4E3XGGRQWKC3LW24Y565B52YJCQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:39.717124420 (45m57.066431696s) [noise=true , address=145.79.227.20:4437   , piece_id=2EGSCSNCQJFXYT54PWEM33SP3RUYICBKKE7IDRQNMZ2ONPM6I5UA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:41.618415745 (45m58.967723031s) [noise=true , address=77.90.31.13:4626     , piece_id=GR3XECVFMZ5E3HDMKK2HLKJREMIOZGWIUI7J3NSK7QGB523G7YJQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.068020890 (45m59.417328166s) [noise=true , address=145.79.227.5:4223    , piece_id=FHAUWKUQAHGYR6ZJHKBCXT3ZPBG42MMJSADUGU7CIFUO3JPPNKKA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:42.150893235 (45m59.500200521s) [noise=true , address=77.90.31.10:4518     , piece_id=76MPB5O5XOR54TOXYCUNK2RBQMWYTWADCBAZV2KQYJ5K7XXDVFMQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:43.703285619 (46m1.052592905s) [noise=true , address=145.79.227.9:4366    , piece_id=5XLRE4OGUCQGZA5UXVNTNTFAMAGGQNHRR4MX23L3362K2DZLDREQ, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.699904003 (46m7.049211289s) [noise=true , address=145.79.227.4:4186    , piece_id=6APC73J3JHGVX3ZNCN3UL5DLHXRYORPQT6AYPFPX6ZQLQIRF5TNA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842045738 (46m7.191353014s) [noise=true , address=145.79.227.7:4294    , piece_id=QFLUZYLIGHO5E3ABEA5LFJPHAKDOF3J55Q4BRF5NXVKDNEHV5DBA, seg_pos={0 0}, upload=668]: Done uploading piece. err: <nil>
Oct 27 15:28:49.842081656 (46m7.191388932s) [seg_pos={0 0}, upload=668]: Segment reached optimal threshold of 13 pieces.
Oct 27 15:28:49.842296495 (46m7.191603781s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo): protocol: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55424->145.79.227.17:4971: use of closed network connection
Oct 27 15:28:49.842306875 (46m7.191614161s) [noise=true , address=145.79.227.17:4971   , piece_id=NK55UPWQTKDWAAMPQTJPPOMZXH5VQGQ3QHJKRNMIRPFLFKQBZZIA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842335229 (46m7.191642535s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gYjGpNnkeFRobb7yhMnm1o9N3H13Uo1fqQrBpRad4uU4rKKm1): protocol: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50900->145.79.227.10:4401: use of closed network connection
Oct 27 15:28:49.842355578 (46m7.191662874s) [noise=true , address=145.79.227.10:4401   , piece_id=SZNSLFKN44YVSUHRXZTF6BBVPYQODYVEBWQF3JHP6ZDA2A723WAA, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842403068 (46m7.191710354s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37220->145.79.227.19:4079: use of closed network connection
Oct 27 15:28:49.842411143 (46m7.191718419s) [noise=true , address=145.79.227.19:4079   , piece_id=QXNYHJS2SBTG2QRH7AKW7NZ6SJG7MFO5DHXCMBZHGT6PJTOZ526Q, seg_pos={0 0}, upload=668]: Long tail cancellation due to optimal threshold
Oct 27 15:28:49.842428026 (46m7.191735312s) [seg_pos={0 0}, upload=668]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil>
Oct 27 15:28:49.842507998 (46m7.191815284s) [upload=668]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
Oct 27 15:28:49.842513478 (46m7.191820764s) [upload=668]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
Oct 27 15:28:50.170126807 (46m7.519434093s) [upload=668]: Done uploading segments...
Oct 27 15:28:50.170148138 (46m7.519455424s) [upload=668]: Upload finished. err: <nil>
Duration: 5.898491923s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)
antiblip: 
1 out of 60 is failed (threshold=3) [.........................X..................................]
duration: 
9.685777924s
FAIL (2025-10-27 16:09:31 UTC) 
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 9.685806709s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote)
antiblip: 
1 out of 60 is failed (threshold=3) [........X...................................................]
duration: 
6.005031943s
FAIL (2025-10-27 16:09:20 UTC) 
orbiter warning: context deadline exceeded
Duration: 6.005058794s edge gateway us-select-1