Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [......................X.........X.....X...X.................] duration:
4.358555389s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:39:01 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.358575779s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.......................X.........X.....X...X................] duration:
4.135302199s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:38:15 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.135344609s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [........................X.........X.....X...X...............] duration:
3.968619741s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:37:47 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 3.968641242s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.........................X.........X.....X...X..............] duration:
4.219447806s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:36:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.219466461s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [..........................X.........X.....X...X.............] duration:
3.969908993s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:35:53 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 3.969981521s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [...........................X.........X.....X...X............] duration:
24.817718034s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:35:04 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 24.817737452s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [............................X.........X.....X...X...........] duration:
23.459130482s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:34:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 23.459157404s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.............................X.........X.....X...X..........] duration:
4.12431527s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:33:11 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.124335999s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [..............................X.........X.....X...X.........] duration:
3.98143169s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:32:52 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 3.981444594s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [...............................X.........X.....X...X........] duration:
4.61214691s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:32:25 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.612158051s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [................................X.........X.....X...X.......] duration:
4.376175564s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:32:00 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.376198288s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.................................X.........X.....X...X......] duration:
4.211540194s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:31:24 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.211562376s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [..................................X.........X.....X...X.....] duration:
3.95685286s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:30:28 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 3.956875244s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [...................................X.........X.....X...X....] duration:
4.285605099s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:29:29 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.285648671s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [....................................X.........X.....X...X...] duration:
6.330171496s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:28:45 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 6.330191084s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.....................................X.........X.....X...X..] duration:
4.243086137s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:28:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 4.243097187s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [......................................X.........X.....X...X.] duration:
58.194621982s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:27:58 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 58.194637862s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
4 out of 60 is failed (threshold=3) [.......................................X.........X.....X...X] duration:
1m0.001313153s trace:
https://jaeger.dev.storj.io/trace/4e48245c0b50a21a
FAIL (2026-05-25 08:26:56 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 25 08:26:46.035810041 (6h22m31.81915013s) [upload=6137]: Starting upload May 25 08:26:46.035848194 (6h22m31.819188293s) [upload=6137]: Done starting upload May 25 08:26:46.035953274 (6h22m31.819293363s) [upload=6137]: Uploading segments... May 25 08:26:46.036030622 (6h22m31.819370721s) [upload=6137]: Got next segment. Inline: false May 25 08:26:46.036060699 (6h22m31.819400798s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 25 08:26:46.036064066 (6h22m31.819404165s) [upload=6137]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 25 08:26:56.790181286 (6h22m42.573521385s) [upload=6137]: Next segment err: context deadline exceeded May 25 08:26:56.790377070 (6h22m42.573717159s) [upload=6137]: Done uploading segments... May 25 08:26:56.790413249 (6h22m42.573753348s) [upload=6137]: Upload finished. err: context deadline exceeded
Duration: 1m0.001327981s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [........................................X.........X.....X...] duration:
4.100970714s trace:
https://jaeger.dev.storj.io/trace/144d39178c68837
FAIL (2026-05-25 08:25:12 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545699663 (6h19m34.329039762s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545982843 (6h19m34.329322942s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545999043 (6h19m34.329339142s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546239642 (6h19m34.329579741s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546381854 (6h19m34.329721953s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546493267 (6h19m34.329833366s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546621762 (6h19m34.329961861s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546713286 (6h19m34.330053385s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546856440 (6h19m34.330196539s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546940450 (6h19m34.330280539s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547078242 (6h19m34.330418331s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547303022 (6h19m34.330643121s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547522991 (6h19m34.33086309s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547787276 (6h19m34.331127375s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.548525324 (6h19m34.331865423s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:49.610210263 (6h19m35.393550362s) [upload=6080]: Next returned nil segment
May 25 08:23:49.610220692 (6h19m35.393560791s) [upload=6080]: Waiting for error group managing segments...
May 25 08:23:49.893558152 (6h19m35.676898251s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894157516 (6h19m35.677497615s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894519837 (6h19m35.677859946s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.900938334 (6h19m35.684278433s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.902493371 (6h19m35.68583347s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.905653560 (6h19m35.688993649s) [noise=true , address=212.102.52.214:4352 , piece_id=ISHVAMUB2GZDIUQOMCLB7QJ77YH46YW6B7ZL27X52DPCIHBFHOBA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.906030189 (6h19m35.689370289s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911170577 (6h19m35.694510676s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911545653 (6h19m35.694885752s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.915089896 (6h19m35.698429995s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.917470267 (6h19m35.700810366s) [noise=true , address=212.102.52.213:4340 , piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.918496214 (6h19m35.701836324s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.952683916 (6h19m35.736024015s) [noise=true , address=212.102.52.216:4413 , piece_id=PSL5INEUF3CADO5QVUIWFHWJRSG4LW733S6AJASDD77GGIXCBGVA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.955937675 (6h19m35.739277774s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.960108533 (6h19m35.743448632s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967209412 (6h19m35.750549511s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967228058 (6h19m35.750568157s) [seg_pos={0 0}, upload=6080]: Satisfied cohort requirements after 16 pieces
May 25 08:23:49.967442637 (6h19m35.750782736s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967449610 (6h19m35.750789709s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967476491 (6h19m35.750816591s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tTPVkbPU79Nu2YCb8QpDDCPuJEYotqgVHPWwcHXC5hAqMnsBd): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967501529 (6h19m35.750841628s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967676664 (6h19m35.751016763s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967701480 (6h19m35.751041579s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967818093 (6h19m35.751158192s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BxSouF8VxKdtKF8TvqPyZFEkAM8qwmgC8ibpwDKpiob8jDp4q): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967824234 (6h19m35.751164333s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967840456 (6h19m35.751180555s) [seg_pos={0 0}, upload=6080]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:23:49.967914356 (6h19m35.751254455s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:23:49.967918575 (6h19m35.751258674s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:23:57.851733943 (6h19m43.635074042s) [upload=6080]: Done uploading segments...
May 25 08:23:57.851802473 (6h19m43.635142563s) [upload=6080]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.100991935s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [.........................................X.........X.....X..] duration:
4.166518022s trace:
https://jaeger.dev.storj.io/trace/144d39178c68837
FAIL (2026-05-25 08:24:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545699663 (6h19m34.329039762s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545982843 (6h19m34.329322942s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545999043 (6h19m34.329339142s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546239642 (6h19m34.329579741s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546381854 (6h19m34.329721953s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546493267 (6h19m34.329833366s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546621762 (6h19m34.329961861s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546713286 (6h19m34.330053385s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546856440 (6h19m34.330196539s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546940450 (6h19m34.330280539s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547078242 (6h19m34.330418331s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547303022 (6h19m34.330643121s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547522991 (6h19m34.33086309s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547787276 (6h19m34.331127375s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.548525324 (6h19m34.331865423s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:49.610210263 (6h19m35.393550362s) [upload=6080]: Next returned nil segment
May 25 08:23:49.610220692 (6h19m35.393560791s) [upload=6080]: Waiting for error group managing segments...
May 25 08:23:49.893558152 (6h19m35.676898251s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894157516 (6h19m35.677497615s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894519837 (6h19m35.677859946s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.900938334 (6h19m35.684278433s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.902493371 (6h19m35.68583347s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.905653560 (6h19m35.688993649s) [noise=true , address=212.102.52.214:4352 , piece_id=ISHVAMUB2GZDIUQOMCLB7QJ77YH46YW6B7ZL27X52DPCIHBFHOBA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.906030189 (6h19m35.689370289s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911170577 (6h19m35.694510676s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911545653 (6h19m35.694885752s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.915089896 (6h19m35.698429995s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.917470267 (6h19m35.700810366s) [noise=true , address=212.102.52.213:4340 , piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.918496214 (6h19m35.701836324s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.952683916 (6h19m35.736024015s) [noise=true , address=212.102.52.216:4413 , piece_id=PSL5INEUF3CADO5QVUIWFHWJRSG4LW733S6AJASDD77GGIXCBGVA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.955937675 (6h19m35.739277774s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.960108533 (6h19m35.743448632s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967209412 (6h19m35.750549511s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967228058 (6h19m35.750568157s) [seg_pos={0 0}, upload=6080]: Satisfied cohort requirements after 16 pieces
May 25 08:23:49.967442637 (6h19m35.750782736s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967449610 (6h19m35.750789709s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967476491 (6h19m35.750816591s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tTPVkbPU79Nu2YCb8QpDDCPuJEYotqgVHPWwcHXC5hAqMnsBd): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967501529 (6h19m35.750841628s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967676664 (6h19m35.751016763s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967701480 (6h19m35.751041579s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967818093 (6h19m35.751158192s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BxSouF8VxKdtKF8TvqPyZFEkAM8qwmgC8ibpwDKpiob8jDp4q): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967824234 (6h19m35.751164333s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967840456 (6h19m35.751180555s) [seg_pos={0 0}, upload=6080]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:23:49.967914356 (6h19m35.751254455s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:23:49.967918575 (6h19m35.751258674s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:23:57.851733943 (6h19m43.635074042s) [upload=6080]: Done uploading segments...
May 25 08:23:57.851802473 (6h19m43.635142563s) [upload=6080]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.166539242s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [..........................................X.........X.....X.] duration:
4.044766805s trace:
https://jaeger.dev.storj.io/trace/144d39178c68837
FAIL (2026-05-25 08:24:02 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545699663 (6h19m34.329039762s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545982843 (6h19m34.329322942s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545999043 (6h19m34.329339142s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546239642 (6h19m34.329579741s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546381854 (6h19m34.329721953s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546493267 (6h19m34.329833366s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546621762 (6h19m34.329961861s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546713286 (6h19m34.330053385s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546856440 (6h19m34.330196539s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546940450 (6h19m34.330280539s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547078242 (6h19m34.330418331s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547303022 (6h19m34.330643121s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547522991 (6h19m34.33086309s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547787276 (6h19m34.331127375s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.548525324 (6h19m34.331865423s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:49.610210263 (6h19m35.393550362s) [upload=6080]: Next returned nil segment
May 25 08:23:49.610220692 (6h19m35.393560791s) [upload=6080]: Waiting for error group managing segments...
May 25 08:23:49.893558152 (6h19m35.676898251s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894157516 (6h19m35.677497615s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894519837 (6h19m35.677859946s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.900938334 (6h19m35.684278433s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.902493371 (6h19m35.68583347s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.905653560 (6h19m35.688993649s) [noise=true , address=212.102.52.214:4352 , piece_id=ISHVAMUB2GZDIUQOMCLB7QJ77YH46YW6B7ZL27X52DPCIHBFHOBA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.906030189 (6h19m35.689370289s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911170577 (6h19m35.694510676s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911545653 (6h19m35.694885752s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.915089896 (6h19m35.698429995s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.917470267 (6h19m35.700810366s) [noise=true , address=212.102.52.213:4340 , piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.918496214 (6h19m35.701836324s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.952683916 (6h19m35.736024015s) [noise=true , address=212.102.52.216:4413 , piece_id=PSL5INEUF3CADO5QVUIWFHWJRSG4LW733S6AJASDD77GGIXCBGVA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.955937675 (6h19m35.739277774s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.960108533 (6h19m35.743448632s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967209412 (6h19m35.750549511s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967228058 (6h19m35.750568157s) [seg_pos={0 0}, upload=6080]: Satisfied cohort requirements after 16 pieces
May 25 08:23:49.967442637 (6h19m35.750782736s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967449610 (6h19m35.750789709s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967476491 (6h19m35.750816591s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tTPVkbPU79Nu2YCb8QpDDCPuJEYotqgVHPWwcHXC5hAqMnsBd): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967501529 (6h19m35.750841628s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967676664 (6h19m35.751016763s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967701480 (6h19m35.751041579s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967818093 (6h19m35.751158192s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BxSouF8VxKdtKF8TvqPyZFEkAM8qwmgC8ibpwDKpiob8jDp4q): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967824234 (6h19m35.751164333s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967840456 (6h19m35.751180555s) [seg_pos={0 0}, upload=6080]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:23:49.967914356 (6h19m35.751254455s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:23:49.967918575 (6h19m35.751258674s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:23:57.851733943 (6h19m43.635074042s) [upload=6080]: Done uploading segments...
May 25 08:23:57.851802473 (6h19m43.635142563s) [upload=6080]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.044779368s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
3 out of 60 is failed (threshold=3) [...........................................X.........X.....X] duration:
1m0.527227931s trace:
https://jaeger.dev.storj.io/trace/144d39178c68837
FAIL (2026-05-25 08:23:57 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
, piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545699663 (6h19m34.329039762s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545982843 (6h19m34.329322942s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.545999043 (6h19m34.329339142s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546239642 (6h19m34.329579741s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546381854 (6h19m34.329721953s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546493267 (6h19m34.329833366s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546621762 (6h19m34.329961861s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546713286 (6h19m34.330053385s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546856440 (6h19m34.330196539s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.546940450 (6h19m34.330280539s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547078242 (6h19m34.330418331s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547303022 (6h19m34.330643121s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547522991 (6h19m34.33086309s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.547787276 (6h19m34.331127375s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:48.548525324 (6h19m34.331865423s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Uploading piece...
May 25 08:23:49.610210263 (6h19m35.393550362s) [upload=6080]: Next returned nil segment
May 25 08:23:49.610220692 (6h19m35.393560791s) [upload=6080]: Waiting for error group managing segments...
May 25 08:23:49.893558152 (6h19m35.676898251s) [noise=true , address=212.102.52.217:4439 , piece_id=VRQR4TSDAML35MPPYILNZ6ATIYOJIU3Y6AJVT4FML3OQV4YVL67A, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894157516 (6h19m35.677497615s) [noise=true , address=212.102.52.216:4401 , piece_id=VG4BEY6K5OTYC57QBRP7NPET4RIELJEIY3INMY6ZJMM4V5APMOZA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.894519837 (6h19m35.677859946s) [noise=true , address=212.102.52.216:4415 , piece_id=LM7R7NBJV26GU6YLRFP4QI57JG7JWSHTML2MCI3MCFNWVYXVXOLA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.900938334 (6h19m35.684278433s) [noise=true , address=212.102.52.214:4360 , piece_id=V4SE7JFM523UWOJQFL2PKEAPD7YOPLGJDDUW3W74A2EDPT6RVDNA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.902493371 (6h19m35.68583347s) [noise=true , address=212.102.52.215:4377 , piece_id=3PAMAK34WT2SKIQ75POHSDZ7VWOCNM7KFNUN6GKRMIXGGQY7ENIA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.905653560 (6h19m35.688993649s) [noise=true , address=212.102.52.214:4352 , piece_id=ISHVAMUB2GZDIUQOMCLB7QJ77YH46YW6B7ZL27X52DPCIHBFHOBA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.906030189 (6h19m35.689370289s) [noise=true , address=212.102.52.217:4431 , piece_id=VORYIQMOJ45GI26RVZ6EY4Z7B3RVYCFFPHSHIZ4ALJ6HR7THQF3Q, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911170577 (6h19m35.694510676s) [noise=true , address=212.102.52.215:4387 , piece_id=5PJ3YK2W6CGET3BG276MXKCYJ6LCM3IZ6YQPXTQSF5X4NJN6JTQQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.911545653 (6h19m35.694885752s) [noise=true , address=212.102.52.214:4355 , piece_id=UBVZE5HWTIQM6EG5WOZ6PFPSEZHYJE7DDI6C4FBV53E6ISAMBQWQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.915089896 (6h19m35.698429995s) [noise=true , address=212.102.52.213:4343 , piece_id=3VVY6UZR5WTLJBINEF5CXD3ECZ6S7MLGVREI3JP54TQUNVZMDGAA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.917470267 (6h19m35.700810366s) [noise=true , address=212.102.52.213:4340 , piece_id=A5FHWH6NTT2U65P26XHA5IQ63XUHMBFNOJ53AYEIJEOSWZMLXFGA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.918496214 (6h19m35.701836324s) [noise=true , address=212.102.52.216:4399 , piece_id=DGIGIB6WRSZBQK6R6AMM5LPDRXJX4AAJLA4OTGXJW2IFNYYDAHMA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.952683916 (6h19m35.736024015s) [noise=true , address=212.102.52.216:4413 , piece_id=PSL5INEUF3CADO5QVUIWFHWJRSG4LW733S6AJASDD77GGIXCBGVA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.955937675 (6h19m35.739277774s) [noise=true , address=212.102.52.213:4332 , piece_id=O6YSGM2X6DEX3HJYBC7MDG7HE3RRTETBWAB5BCASAZSJBFQNY6BQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.960108533 (6h19m35.743448632s) [noise=true , address=212.102.52.217:4425 , piece_id=ND3C5XP42BMVRNU7UWFGEESPXXYZTNJ3IMX3VEFCF6DXYS5XMBSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967209412 (6h19m35.750549511s) [noise=true , address=212.102.52.215:4373 , piece_id=MICF73ABRFCNA2JI4L3GCJNYBIKL6HXOPZZARJBFJJMNMVMGKRHQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: <nil>
May 25 08:23:49.967228058 (6h19m35.750568157s) [seg_pos={0 0}, upload=6080]: Satisfied cohort requirements after 16 pieces
May 25 08:23:49.967442637 (6h19m35.750782736s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ku2quD8AHh79sZcysV979DNjGBVGAazWZM7P5vPeckaKfoBL5): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967449610 (6h19m35.750789709s) [noise=true , address=212.102.52.214:4345 , piece_id=N45FG264KIFAPUPFUKVAD6272NR7RMNS35HOEVUQ7E4AYDPW4WEA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967476491 (6h19m35.750816591s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1tTPVkbPU79Nu2YCb8QpDDCPuJEYotqgVHPWwcHXC5hAqMnsBd): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967501529 (6h19m35.750841628s) [noise=true , address=212.102.52.215:4372 , piece_id=56KVLDTYT6HRIKF7TNFDNLN5J6G5XLM42JUZCRFND77WO44S7EDQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967676664 (6h19m35.751016763s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967701480 (6h19m35.751041579s) [noise=true , address=212.102.52.213:4331 , piece_id=ID3RQUTYQ3LJLO3R2X6A7P5ECAAPJLUKXF4A53NSC7TA2UEEDAYQ, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967818093 (6h19m35.751158192s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BxSouF8VxKdtKF8TvqPyZFEkAM8qwmgC8ibpwDKpiob8jDp4q): protocol: expected piece hash; context canceled; EOF
May 25 08:23:49.967824234 (6h19m35.751164333s) [noise=true , address=212.102.52.217:4418 , piece_id=YSI6YOS2EGIP5SNTLCHG26IYMDB3A6OFP3CEDLKM3AZIQUP5SHSA, seg_pos={0 0}, upload=6080]: Long tail cancellation due to optimal threshold
May 25 08:23:49.967840456 (6h19m35.751180555s) [seg_pos={0 0}, upload=6080]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:23:49.967914356 (6h19m35.751254455s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:23:49.967918575 (6h19m35.751258674s) [upload=6080]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:23:57.851733943 (6h19m43.635074042s) [upload=6080]: Done uploading segments...
May 25 08:23:57.851802473 (6h19m43.635142563s) [upload=6080]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m0.527246736s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............................................X.........X.....] duration:
6.982562613s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:22:18 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 6.982600615s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............................................X.........X....] duration:
3.986056061s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:21:43 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 3.986069607s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............................................X.........X...] duration:
6.21391649s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:20:57 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 6.213937229s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............................................X.........X..] duration:
7.207480607s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:20:15 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 7.207501827s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................................................X.........X.] duration:
3.915761356s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:19:11 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 3.915772698s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................................................X.........X] duration:
1m2.176525036s trace:
https://jaeger.dev.storj.io/trace/3485869bb2e5c930
FAIL (2026-05-25 08:19:07 UTC)
orbiter warning:
operation: sync2.Copy
======ERROR======
uplink: encryption: failed to upload enough pieces (needed at least 16 but got 0)
======LOGS======
POGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Uploading piece...
May 25 08:19:05.117766300 (6h14m50.901106399s) [upload=6006]: Next segment err: context deadline exceeded
May 25 08:19:05.118088345 (6h14m50.901428444s) [noise=true , address=212.102.52.217:4434 , piece_id=KQG6RVZFANV62UKZNJIUJFXESPTAGMCGRXTP7MSDNJZVBKTBWWBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KMzn6eZL5cHZtrCcsMinV1AEroczRHipg53KnfZmVCFW1DsZM, address:212.102.52.217:4434): protocol: read tcp 188.166.225.170:49926->212.102.52.217:4434: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118316710 (6h14m50.901656809s) [noise=true , address=212.102.52.213:4342 , piece_id=NSUN5Z6W6ESOUOOYYF5XPMLYEXH7BFEZRXMGMZKN5ZA3EBOYEUAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1QNtJaVKa9A71zjFz4em41XBwKn6jkUpGNCw279Rd7QCfqPL8K, address:212.102.52.213:4342): protocol: read tcp 188.166.225.170:58960->212.102.52.213:4342: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118459954 (6h14m50.901800053s) [noise=true , address=212.102.52.215:4371 , piece_id=BHW6IUONRCFD34RP77CZHXXGTWB7VH3NAYPINQCBZZQV6NFX5NAQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12ufZvREiiPSyPZ7fvKWVEcCn3E84nCiu6aYxe9n6wP4kKf43hY, address:212.102.52.215:4371): protocol: read tcp 188.166.225.170:34824->212.102.52.215:4371: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118559303 (6h14m50.901899402s) [noise=true , address=212.102.52.217:4427 , piece_id=HX3YLZZMZDXS7HDGFG2GU6LPE3PS2IYNKZHFWPF7KYO4BOBAEZIQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp, address:212.102.52.217:4427): protocol: read tcp 188.166.225.170:48756->212.102.52.217:4427: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118674944 (6h14m50.902015043s) [noise=true , address=212.102.52.214:4364 , piece_id=BTZEZNNNBSVJ7XXEHYULUGRYOB3JF5LL3GFXWPMOYMW4ADYGTFGA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1BHBjpvr3Y4dyKi2KHFy7MpiPzh291UT6X1MMvQLB2ts4xpx6p, address:212.102.52.214:4364): protocol: read tcp 188.166.225.170:50004->212.102.52.214:4364: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118777600 (6h14m50.902117699s) [noise=true , address=212.102.52.216:4415 , piece_id=LDQEIE3SGAJNFVBEEZ27ZFWNKSSS2JZVYEABUOOUKU74HKMOSXBA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1fgavD3HvXpxXs2PV5NacdCzKYqkBXv2oLPquUyDFDk7S93brT, address:212.102.52.216:4415): protocol: read tcp 188.166.225.170:45862->212.102.52.216:4415: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118805453 (6h14m50.902145542s) [noise=true , address=212.102.52.213:4340 , piece_id=5XTBBIAEJ4Y6JJIXUR43KAFEXAE3V6PKXZIORZIXZWLAUKGBM35A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1249tzbqbK8Hmch1EkBQ28yVUNWx11vv7Zf8FtqrkTjEhJtuSGM, address:212.102.52.213:4340): protocol: read tcp 188.166.225.170:51970->212.102.52.213:4340: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118880506 (6h14m50.902220606s) [noise=true , address=212.102.52.213:4329 , piece_id=MIS6NL4X4HTHUPUUXEWD6WWTSWH4JY5NIAGGQOMHKODLPD6Z2HUQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12mH9FnbUFBp3PEhiqXoY1G13G2Qq2kBM13zVQGVHrE3qeYQK2J, address:212.102.52.213:4329): protocol: read tcp 188.166.225.170:58498->212.102.52.213:4329: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118944478 (6h14m50.902284577s) [noise=true , address=212.102.52.213:4331 , piece_id=YGMIW7CVSZYA72PROVBMFOPHPB55CPEAUVCSUYDXPSLNRZQXJSNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1eyfVMwFVb4Q5nbc54PSLebuyWfm2rCgCfGrNc6AovSWqe6Upn, address:212.102.52.213:4331): protocol: read tcp 188.166.225.170:36064->212.102.52.213:4331: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.118979285 (6h14m50.902319384s) [noise=true , address=212.102.52.215:4388 , piece_id=7NUQBSCYYB2NVIDQJYPBVQVDZVBL2PV53RR2GVDXVESORB3OFMCQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF, address:212.102.52.215:4388): protocol: read tcp 188.166.225.170:33096->212.102.52.215:4388: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119057244 (6h14m50.902397333s) [noise=true , address=212.102.52.216:4394 , piece_id=CI2FNS7JIJAR6RFA36TWUBGPPSDZUYG3VBZZIMWSKMW7PRMJPCNA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1WM75xZMY7yQ1ga2AiqsvGZT3uptidF9pFi2T6yoc1jeqGPVcf, address:212.102.52.216:4394): protocol: read tcp 188.166.225.170:44174->212.102.52.216:4394: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119068846 (6h14m50.902408945s) [noise=true , address=212.102.52.215:4370 , piece_id=EQOTUHFSH37DPWFXKX63L7AZZ47NM7TXE6M2ET6HRXIUPUBE5SQQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12wAfNSuNAdVkfMjxBhTsxFpMrKVephQNoWmgtBu4fvUEmGrHtD, address:212.102.52.215:4370): protocol: read tcp 188.166.225.170:59952->212.102.52.215:4370: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119084907 (6h14m50.902425006s) [noise=true , address=212.102.52.216:4399 , piece_id=4VMLOMHEBAWHHFQHWOVGFDAJ5R4ZP7HGWYTNJRT2FB4HP5CTVXEQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12fAGLyjawu1GJ6crrZAEJxVuVwEUHYnUNkvcden4QkK53cVwPy, address:212.102.52.216:4399): protocol: read tcp 188.166.225.170:55390->212.102.52.216:4399: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119224593 (6h14m50.902564692s) [noise=true , address=212.102.52.214:4356 , piece_id=PFUE5DEXLVNJ2O7MW4TJYOKJ5BVT3T3XHPWQZTDGTRQSOLMIQ22A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1263MPtQZy9iB1pxhBYqZ9tgqitVmjm2X7gyehcdSRja9kWfPKd, address:212.102.52.214:4356): protocol: read tcp 188.166.225.170:43210->212.102.52.214:4356: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119381422 (6h14m50.902721521s) [noise=true , address=212.102.52.214:4346 , piece_id=EAV3BFL5IY2MI2AVRPAH6TEIL6YE3ZTFCKRTDJAEMBOOLIMBWUVQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12BA6ZmPnVWhQ86R27YBcNR2f7qDAbjPBadMtYxvzBpDcunYt8F, address:212.102.52.214:4346): protocol: read tcp 188.166.225.170:46098->212.102.52.214:4346: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119588417 (6h14m50.902928506s) [noise=true , address=212.102.52.217:4440 , piece_id=22DGCTHMWQHD5S5GHMB6TACJOLJYTLOAWM77TJIZT3TBG7CCBHCA, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12XWuLksvapgVhPYFPZyPbpJhwQY2YDEvHEFGSkU5KP6qfHLZTk, address:212.102.52.217:4440): protocol: read tcp 188.166.225.170:35096->212.102.52.217:4440: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119734756 (6h14m50.903074845s) [noise=true , address=212.102.52.216:4400 , piece_id=BT3GXTY4X2MIQX6W66KOH5CQURUTFTGAIOA43TR7JJNW2A3XQEHQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1sH5EigMH2sodFcDBup4WLsKZ1jEvQXP9xRRV1HrCEvYCLBZHz, address:212.102.52.216:4400): protocol: read tcp 188.166.225.170:44760->212.102.52.216:4400: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119782047 (6h14m50.903122146s) [noise=true , address=212.102.52.215:4390 , piece_id=JADWLBZHARQID2BRQ4WPOGTD4K45VQU6RS4YZ2ELAEU6M5KZVBRQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK, address:212.102.52.215:4390): protocol: read tcp 188.166.225.170:42360->212.102.52.215:4390: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119881616 (6h14m50.903221716s) [noise=true , address=212.102.52.214:4359 , piece_id=RDGAIHZ6U57RX5RWNIDX5TUYXTKQTS3NEF6UJKF5KOQPNNOBUS3A, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA, address:212.102.52.214:4359): protocol: read tcp 188.166.225.170:55742->212.102.52.214:4359: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119970095 (6h14m50.903310194s) [noise=true , address=212.102.52.217:4431 , piece_id=XCHNQP74SJCY4LD4FZP4I5V3JB72KRLIE22AAERGFVEAYBWBI5VQ, seg_pos={0 0}, upload=6006]: Done uploading piece. err: ecclient: upload failed (node:12GmHVqJ4PzwRM3XoBVXVqdozGWQrn3CMjQ4sBVQAgxQh98pyWG, address:212.102.52.217:4431): protocol: read tcp 188.166.225.170:57354->212.102.52.217:4431: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op
May 25 08:19:05.119993661 (6h14m50.90333376s) [seg_pos={0 0}, upload=6006]: Done waiting for segment. successful: 0 optimal: 16 errs: <nil>
Duration: 1m2.17653779s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
3.999108456s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:17:48 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 3.999123334s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.086427123s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:16:51 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.086449436s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
6.515833086s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:16:26 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 6.515853364s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
4.490176636s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:15:25 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.49020453s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.649600966s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:14:58 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 5.649623669s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
4.488247516s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:13:59 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.488269046s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
18.530535249s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:13:16 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 18.530577949s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.000440464s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:11:56 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 4.000461334s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
10.724000041s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:11:33 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 10.724014668s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m7.093177602s trace:
https://jaeger.dev.storj.io/trace/31eb021aea8c7ba2
FAIL (2026-05-25 08:11:18 UTC)
orbiter warning:
operation: Commit
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
39.402303618s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619373511 (6h6m39.40271361s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619686558 (6h6m39.403026657s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619774877 (6h6m39.403114976s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.619955662 (6h6m39.403295751s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620612025 (6h6m39.403952114s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.620767291 (6h6m39.40410739s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621030253 (6h6m39.404370352s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621604999 (6h6m39.404945098s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.621859354 (6h6m39.405199453s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622651385 (6h6m39.405991484s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.622709035 (6h6m39.406049124s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623186897 (6h6m39.406526996s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.623304832 (6h6m39.406644931s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624087867 (6h6m39.407427956s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:53.624563434 (6h6m39.407903533s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Uploading piece...
May 25 08:10:54.682764935 (6h6m40.466105034s) [upload=5885]: Next returned nil segment
May 25 08:10:54.682782157 (6h6m40.466122256s) [upload=5885]: Waiting for error group managing segments...
May 25 08:10:54.970072321 (6h6m40.75341242s) [noise=true , address=212.102.52.217:4432 , piece_id=M3KH2AL645LONG6BUX7BZCHWPH6XMPPFE4P6GLNIBQROXDKNSEFQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.970376401 (6h6m40.753716501s) [noise=true , address=212.102.52.213:4330 , piece_id=5DGSY2ZPU2ZABJYDI2ITJRW2CLKLWIQFBMVSOOCQM7ZR267PQJXQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.972427345 (6h6m40.755767444s) [noise=true , address=212.102.52.216:4405 , piece_id=AGQNITXZSV5DC5NYXLMMFC6E4XRDBOLBMK7QSETZX6YV7VUC54QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.974783851 (6h6m40.75812395s) [noise=true , address=212.102.52.213:4321 , piece_id=RILH3J4WKEBVX3OWYKYK5ENHTT4VSK4VFQ7KUOK77GS4ZVPXBQIQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978087325 (6h6m40.761427424s) [noise=true , address=212.102.52.213:4329 , piece_id=DRKDJR3IIHFDXLCH2W2YX5GNCWHGL5DPGAGHOR2Q4UHNA6EERRPA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.978247370 (6h6m40.761587469s) [noise=true , address=212.102.52.217:4419 , piece_id=E5WZVCYCVL5E4XZ6WHM4VXXXVZ6W5TZJMNOOGKYBO7LZXAP3ED3A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.979763182 (6h6m40.763103281s) [noise=true , address=212.102.52.217:4417 , piece_id=MS67HUCAYM2THHWMUVPFCHD5HZDRCCFI6PBJJPOVZVWOMGUWVCHQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.980036263 (6h6m40.763376362s) [noise=true , address=212.102.52.216:4411 , piece_id=FDUNY4PWIEDGIAM3LZGJEJEMZEOCGCAS4V2FAMHHPSVD2ALIK6QQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.989106690 (6h6m40.772446789s) [noise=true , address=212.102.52.214:4357 , piece_id=QDQ6LAWQZVCVKBMNGLZOGDWCXQBGPEARZINQTUMFEOQABFZEHBAQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.992088960 (6h6m40.775429069s) [noise=true , address=212.102.52.213:4327 , piece_id=ZI22FHJL54SGNALJL4BCIO2Q4JEJ4BIDQEC5KFHUY4URBG4IJ6IA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:54.995472336 (6h6m40.778812465s) [noise=true , address=212.102.52.215:4388 , piece_id=HS5ZN3MJFH2GQVTRJEARQVUMSFBSWOK6VDOSPKC3INHJSXJ4PC2A, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.023967116 (6h6m40.807307215s) [noise=true , address=212.102.52.214:4353 , piece_id=AHW4SPLOL2J3YQWXYLQZYWWJ4ESJIO2LUIM73WYDFO53EA3PWTCA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.029645951 (6h6m40.81298605s) [noise=true , address=212.102.52.216:4403 , piece_id=75ZP6NBTZGFQIZZXTJ6YGD4F75O65IHBVBD6XXWGFU5SIRFP4TKA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.030767140 (6h6m40.814107239s) [noise=true , address=212.102.52.216:4404 , piece_id=KQTFCE2FND6DSYPFNV23X3TOUH34XFRVGUSOMFFNPE5Y7PBSHUVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.032351523 (6h6m40.815691622s) [noise=true , address=212.102.52.215:4386 , piece_id=BQRRL76YWKJYPUYZGIBYROJVDS7BQFZ23ES3FOOY2CFCCPLIXFUQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034615212 (6h6m40.817955311s) [noise=true , address=212.102.52.214:4364 , piece_id=SDE637S3KFFH3PD46YAJ67BRVK7DZOTI2D6SPGVNDDFLHR36E3CA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: <nil>
May 25 08:10:55.034637615 (6h6m40.817977734s) [seg_pos={0 0}, upload=5885]: Satisfied cohort requirements after 16 pieces
May 25 08:10:55.034924101 (6h6m40.81826421s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hcqjV6TvZLc1B2EKCUncpMAvMAt6Yroqta5rQiXBWeMZwCRkC): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034934121 (6h6m40.81827422s) [noise=true , address=212.102.52.215:4380 , piece_id=AHGSRN7IVD2UO7VMOU74OMMMGU3YJL5T5NABHYHW3U22BF62USVQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.034973115 (6h6m40.818313214s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.034981641 (6h6m40.81832174s) [noise=true , address=212.102.52.214:4359 , piece_id=ST2DGQPXKIS72MXMDJJGWXYZVB6MF7YXT3VG524KFUQ6VJXQAUMA, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035104756 (6h6m40.818444845s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1K3wkvr2DxNuV6iLMnWXW2L12xtjTfjJrSyfzwd6cggnhpQ8xs): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035111399 (6h6m40.818451498s) [noise=true , address=212.102.52.215:4383 , piece_id=WBYHZPYSYDPAJDTPC4EXGWCLVMY54EDKDBY2UT3GCC2D3KZCFDWQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035225897 (6h6m40.818565986s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1cMajrdJow6tXtWWd85iYizrkGjR9cp9kCvCYEKiJGSiLP4PZ8): protocol: expected piece hash; context canceled; EOF
May 25 08:10:55.035230386 (6h6m40.818570485s) [noise=true , address=212.102.52.217:4430 , piece_id=ND5DHIWVT7M3IPPACPCO67SVY7RMX72K7C44LVJQWQAWF4T5YLKQ, seg_pos={0 0}, upload=5885]: Long tail cancellation due to optimal threshold
May 25 08:10:55.035344494 (6h6m40.818684593s) [seg_pos={0 0}, upload=5885]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 08:10:55.035417373 (6h6m40.818757472s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 08:10:55.035422343 (6h6m40.818762442s) [upload=5885]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 08:11:18.402426276 (6h7m4.185766375s) [upload=5885]: Done uploading segments...
May 25 08:11:18.402487933 (6h7m4.185828073s) [upload=5885]: Upload finished. err: metaclient: context deadline exceeded
Duration: 1m7.093196739s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [X...........................................................] duration:
4.298107284s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:49:18 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.298129928s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.X..........................................................] duration:
22.25964246s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:48:38 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 22.259662809s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.042535606s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:47:35 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.042555373s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
4.081769753s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:47:08 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.081791715s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
6.777142926s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:46:36 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 6.777166451s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
4.00838759s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:45:23 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.008409433s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.115091982s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:44:55 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.115118283s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
4.414616253s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:44:37 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.414634768s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........X...................................................] duration:
26.10250998s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:44:32 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 26.102532122s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........X..................................................] duration:
6.010570928s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:43:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 6.010582119s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........X.................................................] duration:
4.151651699s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:42:55 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.151670475s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........X................................................] duration:
4.677922143s trace:
https://jaeger.dev.storj.io/trace/4781335d861d6dbb
FAIL (2026-05-25 06:42:19 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
5 06:06:36.040056527 (4h2m21.823396627s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040316644 (4h2m21.823656743s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040568084 (4h2m21.823908183s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.040810737 (4h2m21.824150836s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041055233 (4h2m21.824395332s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041306904 (4h2m21.824647003s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041553955 (4h2m21.824894054s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.041920755 (4h2m21.825260854s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042257387 (4h2m21.825597486s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042606153 (4h2m21.825946242s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.042990606 (4h2m21.826330705s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043221266 (4h2m21.826561365s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043728965 (4h2m21.827069064s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.043871848 (4h2m21.827211947s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044308541 (4h2m21.82764864s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:36.044651144 (4h2m21.827991243s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Uploading piece...
May 25 06:06:37.108268125 (4h2m22.891608225s) [upload=3866]: Next returned nil segment
May 25 06:06:37.108279057 (4h2m22.891619156s) [upload=3866]: Waiting for error group managing segments...
May 25 06:06:37.389126910 (4h2m23.172467009s) [noise=true , address=212.102.52.215:4375 , piece_id=CPEWCQ5ZCFTULF7OJJYCZPCDWIPQTG6UX3GYIUDKQU4IHDKKT5KA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391635296 (4h2m23.174975395s) [noise=true , address=212.102.52.216:4413 , piece_id=ZZOLRFTT3RLDIZARTH2FWH4OESNO7U5PFQ3VPSJVNATJHVZXPDJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.391804830 (4h2m23.175144929s) [noise=true , address=212.102.52.217:4440 , piece_id=2IHLQGS7M7CTRDMXKBTODIYAJIGKB5R4NRGCX7VLT7GBRJ43QB5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.392934395 (4h2m23.176274494s) [noise=true , address=212.102.52.217:4426 , piece_id=S6IPYBV5WKS55UNUU2NLKZED75NN4Z2I6BIXABZVPT6V4FCW24TA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394811897 (4h2m23.178151996s) [noise=true , address=212.102.52.213:4326 , piece_id=WXKZZNKU5G6TC7KG5HOSFML4A3SDFGPLVVNYDJHR3B6QJWEYRH5Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.394938749 (4h2m23.178278848s) [noise=true , address=212.102.52.217:4428 , piece_id=SCZGE3MAWFZWK767OEXKC5JPENZYMYA5OJEUTICVQNGMMBNZ5QWQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.398257001 (4h2m23.1815971s) [noise=true , address=212.102.52.213:4342 , piece_id=JWRNLXWPUHNPSIVETQ7US5MGK6XKIZ7DWZDPG43O5GGPHUDRQQRQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.400301472 (4h2m23.183641571s) [noise=true , address=212.102.52.214:4360 , piece_id=DE6MZUAV2TXBSS62KGLO6ZLP6ZNMYVGRQQZMTXBW534JRBOTBO2A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.401268066 (4h2m23.184608165s) [noise=true , address=212.102.52.216:4414 , piece_id=CMN5SDXCYRQFR2WP47SU5HWF5F4PENND7WBXQYIMGSNPQ5TXAV6A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.403534320 (4h2m23.186874419s) [noise=true , address=212.102.52.215:4369 , piece_id=YGWIUM3DHBQERTGKE5TZBJIYGELMWFG63GKERBMSYJJZ6NLWRLDQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.407070778 (4h2m23.190410877s) [noise=true , address=212.102.52.214:4367 , piece_id=SGCQTL24NK4BIQL6CY33AW5YYPO2HXWHAERVRM5SXFXOMF7TKHQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.409159473 (4h2m23.192499582s) [noise=true , address=212.102.52.213:4341 , piece_id=3MI3JENAHTA2FX6G72DUKE2N3SYCHSK42C5ZPGDYKNTYERROMDVQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.412127036 (4h2m23.195467135s) [noise=true , address=212.102.52.216:4397 , piece_id=YDQJQLSN7TWH2V45NUJFG4YTHOELISNRFJYW6RVPCDF6I4Y6LIAQ, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422190818 (4h2m23.205530917s) [noise=true , address=212.102.52.216:4407 , piece_id=77EOGE6GIEZXV2YKVLQVRYIKHFO6Z5DJLBLKMEHUOZT5YPCW4W6Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.422198162 (4h2m23.205538261s) [noise=true , address=212.102.52.213:4321 , piece_id=T3N2ISF23D37ZVZ73JRRD22UV4ZTHCIF7R5XAAPHRQ6LMLVYB6XA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449410736 (4h2m23.232750835s) [noise=true , address=212.102.52.215:4384 , piece_id=PSXLV2MKXOPEYHSEJABK6GGHK7L7636THXJOANMDXZ5T5N4R4NJA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: <nil>
May 25 06:06:37.449426625 (4h2m23.232766724s) [seg_pos={0 0}, upload=3866]: Satisfied cohort requirements after 16 pieces
May 25 06:06:37.449579267 (4h2m23.232919366s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12fPSAqiinK7evjPu4BhmWSHtspDDDqAF1bYZpBdS5qV9tEsbXL): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449586260 (4h2m23.232926359s) [noise=true , address=212.102.52.215:4373 , piece_id=DC66AOQL7DLBGTIG3IGG7R7AZQ5JYDIPK5NGCYJFQWJZPETVAJ5A, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449697031 (4h2m23.23303713s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449703814 (4h2m23.233043913s) [noise=true , address=212.102.52.214:4368 , piece_id=6M4ZJ4W7PLVJ4I7ENHYYYW7XFZ7KT3QXJZZSSFTFZC2BJLQJT36Q, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449715637 (4h2m23.233055736s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Qc5vbvWmHz46Ahoqq32hrEXj5bkFxp7ozxghyu9WN5mwtTSJA): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449721699 (4h2m23.233061798s) [noise=true , address=212.102.52.214:4359 , piece_id=V47BYCUAYQ5MUMIBZFQZZX7YCVJDQ7JWL52T7VGGQ7PL3DCPOPOA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449764630 (4h2m23.233104729s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12o9pEghsBzskA6HC7HMDNF7XhJEQFvxBRx95cT7E2RuMxegnNz): protocol: expected piece hash; context canceled; EOF
May 25 06:06:37.449771654 (4h2m23.233111753s) [noise=true , address=212.102.52.217:4437 , piece_id=HGWQ26MRU475VZUHT2V6ZNQTZKCOIVGMDNG2B4LON6TRS55GFRQA, seg_pos={0 0}, upload=3866]: Long tail cancellation due to optimal threshold
May 25 06:06:37.449821429 (4h2m23.233161528s) [seg_pos={0 0}, upload=3866]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 06:06:37.449877716 (4h2m23.233217815s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 06:06:37.449882335 (4h2m23.233222434s) [upload=3866]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 06:06:48.612453934 (4h2m34.395794043s) [upload=3866]: Done uploading segments...
May 25 06:06:48.612473241 (4h2m34.39581335s) [upload=3866]: Upload finished. err: <nil>
Duration: 4.677946148s
core
uplink
uk-select-1
uk1-placement-17
|