Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
2.031686113s
FAIL (2025-09-30 06:53:00 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.03172168s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
2.257570753s
FAIL (2025-09-30 06:51:47 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.257604776s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
2.145806121s
FAIL (2025-09-30 06:51:32 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.145876034s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
2.115114728s
FAIL (2025-09-30 06:51:16 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.115150085s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
2.242474626s
FAIL (2025-09-30 06:50:25 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.242499194s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
2.293589469s
FAIL (2025-09-30 06:49:52 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.293624487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
2.148918908s
FAIL (2025-09-30 06:49:34 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.148955798s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
2.20153513s
FAIL (2025-09-30 06:48:51 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.201560327s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
2.173769018s
FAIL (2025-09-30 06:48:28 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.173804005s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
2.230464953s
FAIL (2025-09-30 06:48:04 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.230523304s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
2.352076278s
FAIL (2025-09-30 06:47:36 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.352116325s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
2.038491411s
FAIL (2025-09-30 06:46:33 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.038545133s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
2.158600112s
FAIL (2025-09-30 06:45:52 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.158635028s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
2.171173663s
FAIL (2025-09-30 06:44:41 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.171200115s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
2.039059533s
FAIL (2025-09-30 06:43:44 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.03909562s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
2.105614521s
FAIL (2025-09-30 06:42:38 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 2.105649548s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1k", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.placement_20.inline)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.001252532s
FAIL (2025-09-30 06:42:31 UTC)
orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 30 06:41:31.801485204 (161h10m17.263577687s) [upload=141704]: Starting upload Sep 30 06:41:31.801592227 (161h10m17.2636847s) [upload=141704]: Done starting upload Sep 30 06:41:31.801664014 (161h10m17.263756497s) [upload=141704]: Uploading segments... Sep 30 06:41:31.801728517 (161h10m17.26382099s) [upload=141704]: Got next segment. Inline: true Sep 30 06:41:31.801734889 (161h10m17.263827372s) [upload=141704]: Waiting for error group managing segments... Sep 30 06:41:31.801831393 (161h10m17.263923876s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectBegin Sep 30 06:41:31.801837905 (161h10m17.263930388s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_SegmentMakeInline Sep 30 06:41:31.801843917 (161h10m17.2639364s) [upload=141704]: Flush batch item: *pb.BatchRequestItem_ObjectCommit Sep 30 06:41:32.097135425 (161h10m17.559227899s) [upload=141704]: Done uploading segments... Sep 30 06:41:32.097156827 (161h10m17.5592493s) [upload=141704]: Upload finished. err: <nil>
Duration: 1m0.00131393s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
7.294200256s
FAIL (2025-09-29 22:28:38 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.29432776s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
7.134244516s
FAIL (2025-09-29 22:28:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.134279323s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [X.X.........................................................] duration:
7.513295335s
FAIL (2025-09-29 22:26:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.513325733s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.X.X........................................................] duration:
7.700010609s
FAIL (2025-09-29 22:25:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.700039704s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..X.X.......................................................] duration:
6.855081161s
FAIL (2025-09-29 22:24:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.855104856s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...X.X......................................................] duration:
9.225830401s
FAIL (2025-09-29 22:24:08 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 9.225854937s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....X.X.....................................................] duration:
7.087221518s
FAIL (2025-09-29 22:23:19 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.087253207s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....X.X....................................................] duration:
7.626590042s
FAIL (2025-09-29 22:22:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.626624497s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......X.X...................................................] duration:
8.319224813s
FAIL (2025-09-29 22:22:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.319258287s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......X.X..................................................] duration:
6.74514343s
FAIL (2025-09-29 22:21:21 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.74518519s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........X.X.................................................] duration:
22.262344396s
FAIL (2025-09-29 22:20:34 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 22.262367681s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........X.X................................................] duration:
7.161723314s
FAIL (2025-09-29 22:19:07 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.161752539s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........X.X...............................................] duration:
8.162761537s
FAIL (2025-09-29 22:18:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.162800251s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........X.X..............................................] duration:
6.804069956s
FAIL (2025-09-29 22:18:00 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.804111544s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............X.X.............................................] duration:
7.695452789s
FAIL (2025-09-29 22:17:13 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.695522171s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............X.X............................................] duration:
7.500112446s
FAIL (2025-09-29 22:16:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.50014141s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............X.X...........................................] duration:
6.4724063s
FAIL (2025-09-29 22:15:30 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.472438552s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...............X.X..........................................] duration:
6.393011688s
FAIL (2025-09-29 22:15:03 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.393046114s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [................X.X.........................................] duration:
7.187190618s
FAIL (2025-09-29 22:14:06 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.187214685s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.................X.X........................................] duration:
7.862318199s
FAIL (2025-09-29 22:13:31 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 7.862349159s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..................X.X.......................................] duration:
6.659623104s
FAIL (2025-09-29 22:13:11 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.659660276s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...................X.X......................................] duration:
6.697552329s
FAIL (2025-09-29 22:12:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.697585212s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [....................X.X.....................................] duration:
6.830307486s
FAIL (2025-09-29 22:11:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.83033479s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.....................X.X....................................] duration:
8.821027726s
FAIL (2025-09-29 22:10:32 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.821057962s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [......................X.X...................................] duration:
6.725042877s
FAIL (2025-09-29 22:09:24 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.725083274s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.......................X.X..................................] duration:
6.963867673s
FAIL (2025-09-29 22:09:05 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.963906336s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [........................X.X.................................] duration:
12.025580785s
FAIL (2025-09-29 22:08:50 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 12.025613448s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.........................X.X................................] duration:
12.240938337s
FAIL (2025-09-29 22:08:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 12.241004804s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..........................X.X...............................] duration:
8.333962572s
FAIL (2025-09-29 22:06:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.33399881s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [...........................X.X..............................] duration:
8.64232443s
FAIL (2025-09-29 22:05:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 8.642353947s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [............................X.X.............................] duration:
13.137879037s
FAIL (2025-09-29 22:04:41 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 13.137909134s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [.............................X.X............................] duration:
6.13040027s
FAIL (2025-09-29 22:03:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.130430448s
core
uplink
us-select-1
|
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)antiblip:
2 out of 60 is failed (threshold=3) [..............................X.X...........................] duration:
6.993340394s
FAIL (2025-09-29 22:02:17 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: failed to upload enough pieces (needed at least 13 but got 12)
======LOGS======
Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload
Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload
Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments...
Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false
Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_ObjectBegin
Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: *pb.BatchRequestItem_SegmentBegin
Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment...
Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment.
Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece...
Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment
Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments...
Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil>
Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF
Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil>
Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments...
Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12)
Duration: 6.993436536s
core
uplink
us-select-1
|