Orbiter History

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................X...............................] duration: 3.586420768s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 19:03:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.586448662s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................X..............................] duration: 3.555665876s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 19:02:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.555696294s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................X.............................] duration: 3.454347151s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 19:01:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.454375484s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................X............................] duration: 3.534187757s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 19:00:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.534215139s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................X...........................] duration: 3.565892508s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:59:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.565920932s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................X..........................] duration: 3.503715331s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:59:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503742073s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................X.........................] duration: 3.510151749s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:58:32 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.510180874s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................X........................] duration: 3.60030129s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:58:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.600330896s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 3.415315595s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:57:35 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.415377092s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 3.489786138s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:56:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.489811125s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.46380857s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:56:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.463836473s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.391830627s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:55:51 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.391857547s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.452714331s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:54:46 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.45273468s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.473480336s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:54:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.473512046s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 3.52445908s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:53:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.524489799s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................X................] duration: 3.466224206s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:53:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.466319787s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [............................................X...............] duration: 3.627413554s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:51:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.627440305s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.............................................X..............] duration: 3.515149045s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:50:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.515255978s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..............................................X.............] duration: 3.498083302s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:49:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.498122876s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...............................................X............] duration: 3.437603694s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:49:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.437647196s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [................................................X...........] duration: 3.436954857s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:48:41 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.436979083s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.................................................X..........] duration: 3.439276559s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:47:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.439315834s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..................................................X.........] duration: 3.498883653s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:46:49 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.498911496s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...................................................X........] duration: 3.437807211s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:46:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.437834413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................................X.......] duration: 3.497491936s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:45:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.49752641s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................................X......] duration: 3.433622891s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:44:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.433662117s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................................X.....] duration: 3.526010708s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:44:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.526048941s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................................X....] duration: 3.439740793s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:43:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.439768497s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................................X...] duration: 3.511217894s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:42:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.511243421s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................................X..] duration: 3.560183894s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:41:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.560290698s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................................X.] duration: 3.502997379s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:40:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.503055899s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [...........................................................X] duration: 5.000622359s trace: https://jaeger.dev.storj.io/trace/73ad65536e3c10a2 FAIL (2025-11-15 18:40:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.000647527s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.312701343s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:51:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.31273563s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.896097125s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:49:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.896129907s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X.X.........................................................] duration: 6.004298594s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:48:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.004331126s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X.X........................................................] duration: 5.849472491s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:47:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.849494211s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration: 6.104973394s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:47:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.10500865s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X.X......................................................] duration: 6.148095145s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:45:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.148128548s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration: 5.960092648s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:44:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.960122995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration: 5.951088111s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:43:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.951123117s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X.X...................................................] duration: 6.120386183s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:43:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.120419718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration: 6.095694863s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:42:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.0957247s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........X.X.................................................] duration: 6.02451631s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:41:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.024537952s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........X.X................................................] duration: 5.858671995s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:41:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.85870142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration: 6.002935433s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:40:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.002963968s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration: 6.017372937s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:39:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.017410969s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............X.X.............................................] duration: 5.805805482s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:38:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.805837734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............X.X............................................] duration: 6.140328539s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:38:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.140358266s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration: 5.886930863s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:36:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.886960711s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration: 5.962868103s trace: https://jaeger.dev.storj.io/trace/3b58e1a240fca649 FAIL (2025-11-15 17:36:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== KYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.402736237 (153h0m18.954206729s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.404744051 (153h0m18.956214502s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.405931692 (153h0m18.957402153s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.407347419 (153h0m18.95881788s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.408581769 (153h0m18.960052231s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.409846437 (153h0m18.961316888s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410225409 (153h0m18.96169587s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.410521122 (153h0m18.961991573s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.412063790 (153h0m18.963534251s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:20.413272761 (153h0m18.964743212s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Uploading piece... Nov 15 17:04:22.175104228 (153h0m20.726574689s) [upload=134743]: Next returned nil segment Nov 15 17:04:22.175120359 (153h0m20.72659082s) [upload=134743]: Waiting for error group managing segments... Nov 15 17:04:22.525313255 (153h0m21.076783716s) [noise=true , address=45.91.116.13:4153 , piece_id=OYKWCD6Y5IVCNOFFY37W2CDH7LLPKJH3VQ3DIKYIOAE56H7FI7HA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.558543764 (153h0m21.110014215s) [noise=true , address=45.91.116.4:4940 , piece_id=XRPR4HVKMTBHXGUKYQYPC5SVXLT7VKGIU2VMO2D62HZY63LTRBKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.559312728 (153h0m21.110783189s) [noise=true , address=45.91.116.6:4973 , piece_id=PHCLA435OL3L2HUC5T5XPN4XMCH46I3WQNX4M2FVRGY6YPXQF7CQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.568814090 (153h0m21.120284561s) [noise=true , address=77.90.31.19:4841 , piece_id=NRGG3WJXIDCHLPPXJF5UBSEKPN567TMJE5J4P43NIHYNX6ETW3PA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:22.611806919 (153h0m21.16327738s) [noise=true , address=45.91.116.7:4010 , piece_id=OUEPWZJ6XORWGVRNJDNNPG7LZ6NLXWXGNIK6PJLAXGN4BKOBOJGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: <nil> Nov 15 17:04:49.406137520 (153h0m47.957607981s) [noise=true , address=145.79.227.24:4581 , piece_id=4RNL2UUVXUQ7VNXJ3TZ3HJ7HOTJP74CG7KBBLGG4EM24MXMNTUGQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:123ib1A8nmVmR272odjhVsUgADYbaMRMiPahSLoZ5eamaV6Qbb1, address:145.79.227.24:4581): protocol: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46588->145.79.227.24:4581: use of closed network connection Nov 15 17:04:49.406210218 (153h0m47.957680679s) [noise=true , address=145.79.227.14:4837 , piece_id=XK3OEKZU7GGVUJIHUNKACQNQHSQ6TACE6RDMKXJFYFEJNXFPTFEA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55178->145.79.227.14:4837: use of closed network connection Nov 15 17:04:49.406419527 (153h0m47.957889988s) [noise=true , address=145.79.227.23:4546 , piece_id=IT5MK464K332B27EDTBCTEZVPVNQPGJAJMRTENHKDSHV2R7GHXPQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1ZMTFpqfMjeVWXJuvsGdbGkMtGQn6axUZFoU1N9sK6fnbPcMy5, address:145.79.227.23:4546): protocol: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59540->145.79.227.23:4546: use of closed network connection Nov 15 17:04:49.406473238 (153h0m47.957943689s) [noise=true , address=145.79.227.32:4836 , piece_id=7HUMFSGYWJZQDIHSFTCYB2WLGFDXK6XN34G6422RQOAKHQHHRTKQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1jic2LawZWwB7J3oFbhRSM2YMurEqQQJM8UEEwWj25h3PxMugR, address:145.79.227.32:4836): protocol: read tcp 188.166.225.170:33336->145.79.227.32:4836: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406604979 (153h0m47.95807543s) [noise=true , address=145.79.227.10:4402 , piece_id=QJ7J3TY6QURQVIC7ROYIVEPLQKDSUBATHESEK5GHCM3LZF3LEMUA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12CMKNUtKvidnZpTfnYPxDCPUdvaSUs5mAx9oFFd2bWfuocTT4u, address:145.79.227.10:4402): protocol: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40930->145.79.227.10:4402: use of closed network connection Nov 15 17:04:49.406673640 (153h0m47.958144101s) [noise=true , address=145.79.227.9:4366 , piece_id=37BKKXN6NGTG4BVKXJKFHBTFXGWESVYBENAN5DBG3LAP3LUM3R5A, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1evHXReGUBYNCZaGnZ2PwmN4xZH1ABzorfhP7w61sHq4EK8Cpy, address:145.79.227.9:4366): protocol: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37090->145.79.227.9:4366: use of closed network connection Nov 15 17:04:49.406773711 (153h0m47.958244162s) [noise=true , address=145.79.227.28:4727 , piece_id=DQQTMDTW64S3JI7A2OA3BEWHN5VHT3APLQ7KN3IUPAFP3L4BCYZA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12LWaXPeH6XghWc64oeseaM52Tk2PQZWDDj1nVnYikHGz7PwEQ3, address:145.79.227.28:4727): protocol: read tcp 188.166.225.170:58720->145.79.227.28:4727: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406798208 (153h0m47.958268669s) [noise=true , address=145.79.227.18:4005 , piece_id=RAZTIYP2DVOP5HJZUETHO44GKARZ6E2MC7ILNG2JLSD7UOVLDJXA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55558->145.79.227.18:4005: use of closed network connection Nov 15 17:04:49.406872709 (153h0m47.958343161s) [noise=true , address=145.79.227.7:4295 , piece_id=VHGGSHPQMAZ3PJ4S55NWWIESM2US4H2FJCSOYUFLMVOUG5IVWKKA, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:38702->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.406964343 (153h0m47.958434794s) [noise=true , address=145.79.227.11:4731 , piece_id=TTSYRUNAC5QQ3NPCI2TWTMDWAY244IQMZ76CH647VJQDDL2BNXAQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: read tcp 188.166.225.170:45034->145.79.227.11:4731: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407084122 (153h0m47.958554573s) [noise=true , address=145.79.227.17:4972 , piece_id=U5IG7O26H62SOZORBFIL5JKWV3O7FEO7JHOEQNRPZ3NDIN2MRCNQ, seg_pos={0 0}, upload=134743]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:57568->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Nov 15 17:04:49.407104120 (153h0m47.958574572s) [seg_pos={0 0}, upload=134743]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Nov 15 17:04:49.673197752 (153h0m48.224668203s) [upload=134743]: Done uploading segments... Nov 15 17:04:49.673234111 (153h0m48.224704562s) [upload=134743]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.962987161s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)